2025-04-02 03:31:18,356 [ 447130 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:53, check_args_and_update_paths) 2025-04-02 03:31:18,356 [ 447130 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:97, check_args_and_update_paths) 2025-04-02 03:31:18,356 [ 447130 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:108, check_args_and_update_paths) 2025-04-02 03:31:18,356 [ 447130 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:110, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_afdvau --privileged --dns-search='.' --memory=30709026816 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=5dc43a6382f0 -e DOCKER_BASE_TAG=8b2301119731 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_part_log_table/test.py::test_config_disk_name_test test_part_log_table/test.py::test_config_with_non_standard_part_log test_part_log_table/test.py::test_config_with_standard_part_log test_part_log_table/test.py::test_config_without_part_log test_part_log_table/test.py::test_part_log_contains_partition test_partition/test.py::test_attach_check_all_parts test_partition/test.py::test_cannot_attach_active_part test_partition/test.py::test_detached_part_dir_exists test_partition/test.py::test_drop_detached_parts test_partition/test.py::test_make_clone_in_detached test_partition/test.py::test_partition_complex test_partition/test.py::test_partition_simple test_partition/test.py::test_system_detached_parts test_postgresql_replica_database_engine_1/test.py::test_abrupt_connection_loss_while_heavy_replication test_postgresql_replica_database_engine_1/test.py::test_abrupt_server_restart_while_heavy_replication test_postgresql_replica_database_engine_1/test.py::test_changing_replica_identity_value test_postgresql_replica_database_engine_1/test.py::test_clickhouse_restart test_postgresql_replica_database_engine_1/test.py::test_concurrent_transactions test_postgresql_replica_database_engine_1/test.py::test_different_data_types test_postgresql_replica_database_engine_1/test.py::test_drop_database_while_replication_startup_not_finished test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_all_database_tables test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_subset_of_database_tables test_postgresql_replica_database_engine_1/test.py::test_many_concurrent_queries test_postgresql_replica_database_engine_1/test.py::test_multiple_databases test_postgresql_replica_database_engine_1/test.py::test_quoting_1 test_postgresql_replica_database_engine_1/test.py::test_quoting_2 test_postgresql_replica_database_engine_1/test.py::test_replica_identity_index test_postgresql_replica_database_engine_1/test.py::test_replicating_dml test_postgresql_replica_database_engine_1/test.py::test_restart_server_while_replication_startup_not_finished test_postgresql_replica_database_engine_1/test.py::test_single_transaction test_postgresql_replica_database_engine_1/test.py::test_table_schema_changes test_postgresql_replica_database_engine_1/test.py::test_user_managed_slots test_postgresql_replica_database_engine_1/test.py::test_virtual_columns test_quota/test.py::test_add_remove_interval test_quota/test.py::test_add_remove_quota test_quota/test.py::test_consumption_of_show_clusters test_quota/test.py::test_consumption_of_show_databases test_quota/test.py::test_consumption_of_show_privileges test_quota/test.py::test_consumption_of_show_processlist test_quota/test.py::test_consumption_of_show_tables test_quota/test.py::test_dcl_introspection test_quota/test.py::test_dcl_management test_quota/test.py::test_exceed_quota test_quota/test.py::test_query_inserts test_quota/test.py::test_quota_from_users_xml test_quota/test.py::test_reload_users_xml_by_timer test_quota/test.py::test_simpliest_quota test_quota/test.py::test_tracking_quota test_quota/test.py::test_users_xml_is_readonly test_reload_client_certificate/test.py::test_correct_cn_cert test_reload_client_certificate/test.py::test_wrong_cn_cert test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop test_reload_query_masking_rules/test.py::test_reload_query_masking_rules test_reload_zookeeper/test.py::test_reload_zookeeper test_reloading_storage_configuration/test.py::test_add_disk test_reloading_storage_configuration/test.py::test_add_disk_to_policy test_reloading_storage_configuration/test.py::test_add_disk_to_separate_config test_reloading_storage_configuration/test.py::test_add_policy test_reloading_storage_configuration/test.py::test_add_volume_to_policy test_reloading_storage_configuration/test.py::test_new_policy_works test_reloading_storage_configuration/test.py::test_remove_disk test_reloading_storage_configuration/test.py::test_remove_disk_from_policy test_reloading_storage_configuration/test.py::test_remove_policy test_reloading_storage_configuration/test.py::test_remove_volume_from_policy test_reloading_storage_configuration/test.py::test_update_disk test_remote_prewhere/test.py::test_remote test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper test_rename_column/test.py::test_rename_distributed test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select test_rename_column/test.py::test_rename_parallel test_rename_column/test.py::test_rename_parallel_same_node test_rename_column/test.py::test_rename_with_parallel_insert test_rename_column/test.py::test_rename_with_parallel_merges test_rename_column/test.py::test_rename_with_parallel_select test_rename_column/test.py::test_rename_with_parallel_slow_insert test_rename_column/test.py::test_rename_with_parallel_ttl_delete test_rename_column/test.py::test_rename_with_parallel_ttl_move test_replicated_database_alter_modify_order_by/test.py::test_alter_modify_order_by test_replicated_database_cluster_groups/test.py::test_cluster_groups test_replicated_engine_arguments/test.py::test_replicated_engine_with_arguments test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch test_replicated_merge_tree_compatibility/test.py::test_replicated_merge_tree_defaults_compatibility test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings test_replicated_merge_tree_encrypted_disk/test.py::test_different_keys test_replicated_merge_tree_encrypted_disk/test.py::test_same_keys test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids test_replicated_merge_tree_encryption_codec/test.py::test_different_keys test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces test_replicated_merge_tree_encryption_codec/test.py::test_same_keys test_replicated_merge_tree_s3_zero_copy/test.py::test_drop_table 'test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[0-16]' 'test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[8192-12]' test_replicated_merge_tree_s3_zero_copy/test.py::test_s3_check_restore test_replicated_mutations/test.py::test_mutations 'test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes0]' 'test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes1]' -vvv -ss" altinityinfra/integration-tests-runner:2165613c5fcd '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket= rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: timeout-2.3.1, repeat-0.9.3, order-1.0.0, reportlog-0.4.0, xdist-3.5.0, random-order-1.1.1 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_postgresql_replica_database_engine_1/test.py::test_abrupt_connection_loss_while_heavy_replication test_partition/test.py::test_attach_check_all_parts Command:[docker ps | wc -l] Command:[docker ps | wc -l] Command:[docker ps | wc -l] Command:[docker ps | wc -l] Command:[docker ps | wc -l] Command:[docker ps | wc -l] Command:[docker ps | wc -l] Command:[docker ps | wc -l] Command:[docker ps | wc -l] Command:[docker ps | wc -l] test_rename_column/test.py::test_rename_distributed test_part_log_table/test.py::test_config_disk_name_test test_quota/test.py::test_add_remove_interval test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table test_replicated_merge_tree_s3_zero_copy/test.py::test_drop_table test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids test_replicated_mutations/test.py::test_mutations test_reloading_storage_configuration/test.py::test_add_disk Stdout:1 No running containers Pruning Docker networks Stdout:1 Command:[docker network prune --force] No running containers Stdout:1 Pruning Docker networks Command:[docker network prune --force] No running containers Pruning Docker networks Command:[docker network prune --force] Stdout:1 No running containers Pruning Docker networks Command:[docker network prune --force] Stdout:1 No running containers Pruning Docker networks Command:[docker network prune --force] Stdout:1 Stdout:1 Stdout:1 No running containers No running containers Pruning Docker networks Pruning Docker networks No running containers Command:[docker network prune --force] Command:[docker network prune --force] Pruning Docker networks Command:[docker network prune --force] Stdout:1 No running containers Pruning Docker networks Command:[docker network prune --force] Stdout:1 No running containers Pruning Docker networks Command:[docker network prune --force] Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] Stdout:net.ipv4.ip_local_port_range = 55000 65535 Running tests in /ClickHouse/tests/integration/test_rename_column/test.py Cluster start called. is_up=False Stdout:net.ipv4.ip_local_port_range = 55000 65535 Stdout:net.ipv4.ip_local_port_range = 55000 65535 Running tests in /ClickHouse/tests/integration/test_partition/test.py Stdout:net.ipv4.ip_local_port_range = 55000 65535 Cluster start called. is_up=False Running tests in /ClickHouse/tests/integration/test_quota/test.py Stdout:net.ipv4.ip_local_port_range = 55000 65535 ENV DOCKER_KERBEROS_KDC_TAG 9391ecdee8d7 Stdout:net.ipv4.ip_local_port_range = 55000 65535 Cluster start called. is_up=False ENV CLICKHOUSE_TESTS_SERVER_BIN_PATH /clickhouse ENV MSAN_OPTIONS abort_on_error=1 poison_in_dtor=1 ENV JAVA_TOOL_OPTIONS -Djdk.attach.allowAttachSelf=true ENV TSAN_OPTIONS halt_on_error=1 abort_on_error=1 history_size=7 memory_limit_mb=46080 second_deadlock_stack=1 ENV HOSTNAME 6cc0b43913bc Running tests in /ClickHouse/tests/integration/test_reloading_storage_configuration/test.py ENV SHLVL 0 ENV HOME /root Running tests in /ClickHouse/tests/integration/test_part_log_table/test.py ENV OLDPWD / Cluster start called. is_up=False ENV DOCKER_HELPER_TAG 5dc43a6382f0 ENV PYTHONUNBUFFERED 1 Cluster start called. is_up=False Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] ENV DOCKER_PYTHON_BOTTLE_TAG caad4729259e ENV UBSAN_OPTIONS print_stacktrace=1 ENV PYTEST_ADDOPTS --dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_part_log_table/test.py::test_config_disk_name_test test_part_log_table/test.py::test_config_with_non_standard_part_log test_part_log_table/test.py::test_config_with_standard_part_log test_part_log_table/test.py::test_config_without_part_log test_part_log_table/test.py::test_part_log_contains_partition test_partition/test.py::test_attach_check_all_parts test_partition/test.py::test_cannot_attach_active_part test_partition/test.py::test_detached_part_dir_exists test_partition/test.py::test_drop_detached_parts test_partition/test.py::test_make_clone_in_detached test_partition/test.py::test_partition_complex test_partition/test.py::test_partition_simple test_partition/test.py::test_system_detached_parts test_postgresql_replica_database_engine_1/test.py::test_abrupt_connection_loss_while_heavy_replication test_postgresql_replica_database_engine_1/test.py::test_abrupt_server_restart_while_heavy_replication test_postgresql_replica_database_engine_1/test.py::test_changing_replica_identity_value test_postgresql_replica_database_engine_1/test.py::test_clickhouse_restart test_postgresql_replica_database_engine_1/test.py::test_concurrent_transactions test_postgresql_replica_database_engine_1/test.py::test_different_data_types test_postgresql_replica_database_engine_1/test.py::test_drop_database_while_replication_startup_not_finished test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_all_database_tables test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_subset_of_database_tables test_postgresql_replica_database_engine_1/test.py::test_many_concurrent_queries test_postgresql_replica_database_engine_1/test.py::test_multiple_databases test_postgresql_replica_database_engine_1/test.py::test_quoting_1 test_postgresql_replica_database_engine_1/test.py::test_quoting_2 test_postgresql_replica_database_engine_1/test.py::test_replica_identity_index test_postgresql_replica_database_engine_1/test.py::test_replicating_dml test_postgresql_replica_database_engine_1/test.py::test_restart_server_while_replication_startup_not_finished test_postgresql_replica_database_engine_1/test.py::test_single_transaction test_postgresql_replica_database_engine_1/test.py::test_table_schema_changes test_postgresql_replica_database_engine_1/test.py::test_user_managed_slots test_postgresql_replica_database_engine_1/test.py::test_virtual_columns test_quota/test.py::test_add_remove_interval test_quota/test.py::test_add_remove_quota test_quota/test.py::test_consumption_of_show_clusters test_quota/test.py::test_consumption_of_show_databases test_quota/test.py::test_consumption_of_show_privileges test_quota/test.py::test_consumption_of_show_processlist test_quota/test.py::test_consumption_of_show_tables test_quota/test.py::test_dcl_introspection test_quota/test.py::test_dcl_management test_quota/test.py::test_exceed_quota test_quota/test.py::test_query_inserts test_quota/test.py::test_quota_from_users_xml test_quota/test.py::test_reload_users_xml_by_timer test_quota/test.py::test_simpliest_quota test_quota/test.py::test_tracking_quota test_quota/test.py::test_users_xml_is_readonly test_reload_client_certificate/test.py::test_correct_cn_cert test_reload_client_certificate/test.py::test_wrong_cn_cert test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop test_reload_query_masking_rules/test.py::test_reload_query_masking_rules test_reload_zookeeper/test.py::test_reload_zookeeper test_reloading_storage_configuration/test.py::test_add_disk test_reloading_storage_configuration/test.py::test_add_disk_to_policy test_reloading_storage_configuration/test.py::test_add_disk_to_separate_config test_reloading_storage_configuration/test.py::test_add_policy test_reloading_storage_configuration/test.py::test_add_volume_to_policy test_reloading_storage_configuration/test.py::test_new_policy_works test_reloading_storage_configuration/test.py::test_remove_disk test_reloading_storage_configuration/test.py::test_remove_disk_from_policy test_reloading_storage_configuration/test.py::test_remove_policy test_reloading_storage_configuration/test.py::test_remove_volume_from_policy test_reloading_storage_configuration/test.py::test_update_disk test_remote_prewhere/test.py::test_remote test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper test_rename_column/test.py::test_rename_distributed test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select test_rename_column/test.py::test_rename_parallel test_rename_column/test.py::test_rename_parallel_same_node test_rename_column/test.py::test_rename_with_parallel_insert test_rename_column/test.py::test_rename_with_parallel_merges test_rename_column/test.py::test_rename_with_parallel_select test_rename_column/test.py::test_rename_with_parallel_slow_insert test_rename_column/test.py::test_rename_with_parallel_ttl_delete test_rename_column/test.py::test_rename_with_parallel_ttl_move test_replicated_database_alter_modify_order_by/test.py::test_alter_modify_order_by test_replicated_database_cluster_groups/test.py::test_cluster_groups test_replicated_engine_arguments/test.py::test_replicated_engine_with_arguments test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch test_replicated_merge_tree_compatibility/test.py::test_replicated_merge_tree_defaults_compatibility test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings test_replicated_merge_tree_encrypted_disk/test.py::test_different_keys test_replicated_merge_tree_encrypted_disk/test.py::test_same_keys test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids test_replicated_merge_tree_encryption_codec/test.py::test_different_keys test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces test_replicated_merge_tree_encryption_codec/test.py::test_same_keys test_replicated_merge_tree_s3_zero_copy/test.py::test_drop_table 'test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[0-16]' 'test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[8192-12]' test_replicated_merge_tree_s3_zero_copy/test.py::test_s3_check_restore test_replicated_mutations/test.py::test_mutations 'test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes0]' 'test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes1]' -vvv -ss ENV CLICKHOUSE_LIBRARY_BRIDGE_BINARY_PATH /clickhouse-library-bridge ENV COMPOSE_HTTP_TIMEOUT 600 ENV DOCKER_MYSQL_PHP_CLIENT_TAG 88be89c1e3b6 ENV DOCKER_DOTNET_CLIENT_TAG 11de0b29a15d ENV CLICKHOUSE_TESTS_CLIENT_BIN_PATH /clickhouse ENV DOCKER_MYSQL_JS_CLIENT_TAG 41ba7c2ec2a1 ENV PATH /spark-3.3.2-bin-hadoop3/bin:/opt/gdb/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin ENV DOCKER_KERBERIZED_HADOOP_TAG latest ENV DOCKER_CHANNEL stable ENV DOCKER_CLIENT_TIMEOUT 300 ENV DOCKER_POSTGRESQL_JAVA_CLIENT_TAG a4eff5c7f4d6 ENV DOCKER_NGINX_DAV_TAG b55ac9cd7519 ENV DOCKER_MYSQL_GOLANG_CLIENT_TAG 9bec2a638e6e ENV PWD /ClickHouse/tests/integration ENV DOCKER_MYSQL_JAVA_CLIENT_TAG 766bff31cfe4 ENV CLICKHOUSE_ODBC_BRIDGE_BINARY_PATH /clickhouse-odbc-bridge ENV CLICKHOUSE_TESTS_BASE_CONFIG_DIR /clickhouse-config Stdout:net.ipv4.ip_local_port_range = 55000 65535 ENV TZ Etc/UTC Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] ENV JAVA_PATH /usr/lib/jvm/java-11-openjdk-amd64/bin/java ENV DOCKER_BASE_TAG 8b2301119731 ENV SPARK_HOME /spark-3.3.2-bin-hadoop3 ENV LC_CTYPE C.UTF-8 ENV INTEGRATION_TESTS_RUN_ID 0 Running tests in /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/test.py ENV WORKER_FREE_PORTS 30400 30401 30402 30403 30404 30405 30406 30407 30408 30409 30410 30411 30412 30413 30414 30415 30416 30417 30418 30419 30420 30421 30422 30423 30424 30425 30426 30427 30428 30429 30430 30431 30432 30433 30434 30435 30436 30437 30438 30439 30440 30441 30442 30443 30444 30445 30446 30447 30448 30449 ENV PYTEST_XDIST_TESTRUNUID 769df26149b54feab59d153d6993d7ee ENV PYTEST_XDIST_WORKER gw8 ENV PYTEST_XDIST_WORKER_COUNT 10 Cluster start called. is_up=False ENV PYTEST_CURRENT_TEST test_replicated_merge_tree_s3_zero_copy/test.py::test_drop_table (setup) CLUSTER INIT base_config_dir:/clickhouse-config Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log Setup Keeper Cluster name: project_name:roottestreplicatedmergetrees3zerocopy-gw8. Added instance name:node1 tag:8b2301119731 base_cmd:['docker', 'compose', '--env-file', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env', '--project-name', 'roottestreplicatedmergetrees3zerocopy-gw8', '--file', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log Cluster name: project_name:roottestreplicatedmergetrees3zerocopy-gw8. Added instance name:node2 tag:8b2301119731 base_cmd:['docker', 'compose', '--env-file', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env', '--project-name', 'roottestreplicatedmergetrees3zerocopy-gw8', '--file', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log Cluster name: project_name:roottestreplicatedmergetrees3zerocopy-gw8. Added instance name:node3 tag:8b2301119731 base_cmd:['docker', 'compose', '--env-file', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env', '--project-name', 'roottestreplicatedmergetrees3zerocopy-gw8', '--file', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ Starting cluster... Running tests in /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/test.py Cluster start called. is_up=False Stdout:net.ipv4.ip_local_port_range = 55000 65535 Stdout:net.ipv4.ip_local_port_range = 55000 65535 Running tests in /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/test.py Running tests in /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/test.py Cluster start called. is_up=False Cluster start called. is_up=False Command:[sysctl net.ipv4.ip_local_port_range='55000 65535'] Stdout:net.ipv4.ip_local_port_range = 55000 65535 Running tests in /ClickHouse/tests/integration/test_replicated_mutations/test.py Cluster start called. is_up=False Docker networks for project roottestrenamecolumn-gw3 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestquota-gw1 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestpartlogtable-gw4 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestreloadingstorageconfiguration-gw2 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestreplicatedmergetreeencryptioncodec-gw7 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestreplicatedmergetrees3zerocopy-gw8 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestpartition-gw5 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestreplicatedfetchesbandwidth-gw6 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestreplicatedmutations-gw9 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestpostgresqlreplicadatabaseengine1-gw0 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestrenamecolumn-gw3 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestreloadingstorageconfiguration-gw2 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestreplicatedmergetrees3zerocopy-gw8 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestquota-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestpartlogtable-gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestpartition-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestreplicatedmergetreeencryptioncodec-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestreplicatedmutations-gw9 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestreplicatedfetchesbandwidth-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestpostgresqlreplicadatabaseengine1-gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreloadingstorageconfiguration-gw2 are DRIVER VOLUME NAME Cleanup called Docker volumes for project roottestquota-gw1 are DRIVER VOLUME NAME Cleanup called Docker volumes for project roottestreplicatedmergetrees3zerocopy-gw8 are DRIVER VOLUME NAME Cleanup called Docker volumes for project roottestrenamecolumn-gw3 are DRIVER VOLUME NAME Cleanup called Docker volumes for project roottestreplicatedmergetreeencryptioncodec-gw7 are DRIVER VOLUME NAME Cleanup called Docker volumes for project roottestpartition-gw5 are DRIVER VOLUME NAME Cleanup called Docker volumes for project roottestpartlogtable-gw4 are DRIVER VOLUME NAME Cleanup called Docker volumes for project roottestreplicatedmutations-gw9 are DRIVER VOLUME NAME Cleanup called Docker volumes for project roottestreplicatedfetchesbandwidth-gw6 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestreloadingstorageconfiguration-gw2 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestrenamecolumn-gw3 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestquota-gw1 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestpartition-gw5 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestreplicatedmergetrees3zerocopy-gw8 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestreplicatedmutations-gw9 are NETWORK ID NAME DRIVER SCOPE Docker volumes for project roottestpostgresqlreplicadatabaseengine1-gw0 are DRIVER VOLUME NAME Docker networks for project roottestreplicatedmergetreeencryptioncodec-gw7 are NETWORK ID NAME DRIVER SCOPE Cleanup called Docker networks for project roottestreplicatedfetchesbandwidth-gw6 are NETWORK ID NAME DRIVER SCOPE Docker networks for project roottestpartlogtable-gw4 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestrenamecolumn-gw3 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestquota-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestpartition-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestreloadingstorageconfiguration-gw2 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestreplicatedfetchesbandwidth-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestpartlogtable-gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestreplicatedmergetreeencryptioncodec-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestreplicatedmutations-gw9 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker containers for project roottestreplicatedmergetrees3zerocopy-gw8 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker networks for project roottestpostgresqlreplicadatabaseengine1-gw0 are NETWORK ID NAME DRIVER SCOPE Docker volumes for project roottestrenamecolumn-gw3 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestrenamecolumn-gw3-.*-1$' --format '{{.ID}}:{{.Names}}'] Docker volumes for project roottestquota-gw1 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestquota-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] Docker volumes for project roottestpartition-gw5 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestpartition-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] Docker volumes for project roottestreloadingstorageconfiguration-gw2 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreloadingstorageconfiguration-gw2-.*-1$' --format '{{.ID}}:{{.Names}}'] Docker volumes for project roottestreplicatedfetchesbandwidth-gw6 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedfetchesbandwidth-gw6-.*-1$' --format '{{.ID}}:{{.Names}}'] Docker volumes for project roottestreplicatedmutations-gw9 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmutations-gw9-.*-1$' --format '{{.ID}}:{{.Names}}'] Docker containers for project roottestpostgresqlreplicadatabaseengine1-gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Unstopped containers: {} No running containers for project: roottestrenamecolumn-gw3 Trying to prune unused networks... Docker volumes for project roottestreplicatedmergetreeencryptioncodec-gw7 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmergetreeencryptioncodec-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] Docker volumes for project roottestreplicatedmergetrees3zerocopy-gw8 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmergetrees3zerocopy-gw8-.*-1$' --format '{{.ID}}:{{.Names}}'] Docker volumes for project roottestpartlogtable-gw4 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestpartlogtable-gw4-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestquota-gw1 Trying to prune unused networks... Unstopped containers: {} No running containers for project: roottestreloadingstorageconfiguration-gw2 Trying to prune unused networks... Unstopped containers: {} No running containers for project: roottestpartition-gw5 Trying to prune unused networks... Unstopped containers: {} Unstopped containers: {} No running containers for project: roottestreplicatedfetchesbandwidth-gw6 Trying to prune unused networks... No running containers for project: roottestreplicatedmutations-gw9 Trying to prune unused networks... Docker volumes for project roottestpostgresqlreplicadatabaseengine1-gw0 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestpostgresqlreplicadatabaseengine1-gw0-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicatedmergetrees3zerocopy-gw8 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Unstopped containers: {} No running containers for project: roottestreplicatedmergetreeencryptioncodec-gw7 Trying to prune unused networks... Unstopped containers: {} No running containers for project: roottestpartlogtable-gw4 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Trying to prune unused images... Command:[docker image prune -f] Trying to prune unused images... Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused images... Trying to prune unused volumes... Command:[docker image prune -f] Command:[docker image prune -f] Command:[docker volume ls | wc -l] Unstopped containers: {} No running containers for project: roottestpostgresqlreplicadatabaseengine1-gw0 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Trying to prune unused images... Command:[docker image prune -f] Trying to prune unused images... Command:[docker image prune -f] Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stderr:Error response from daemon: a prune operation is already running Exitcode:1 Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Trying to prune unused images... Stderr:Error response from daemon: a prune operation is already running Stdout:Total reclaimed space: 0B Command:[docker image prune -f] Images pruned Exitcode:1 Trying to prune unused volumes... Trying to prune unused volumes... Command:[docker volume ls | wc -l] Command:[docker volume ls | wc -l] Stderr:Error response from daemon: a prune operation is already running Exitcode:1 Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:1 Volumes pruned: 1 Setup directory for instance: node1 Create directory for configuration generated in this helper Create directory for common tests configuration Stdout:Total reclaimed space: 0B Copy common configuration from helpers Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_rename_column/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/instant_moves.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/part_log.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/zookeeper_session_timeout.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/storage_configuration.xml'] to /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node1/database Setup logs dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node1/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup directory for instance: node2 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_rename_column/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/instant_moves.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/part_log.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/zookeeper_session_timeout.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/storage_configuration.xml'] to /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node2/configs/config.d Setup database dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node2/database Setup logs dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node2/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Stdout:1 Setup directory for instance: node3 Volumes pruned: 1 Setup directory for instance: instance Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_rename_column/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/instant_moves.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/part_log.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/zookeeper_session_timeout.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/storage_configuration.xml'] to /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node3/configs/config.d Copy custom test config files ['/ClickHouse/tests/integration/test_partition/configs/testkeeper.xml'] to /ClickHouse/tests/integration/test_partition/_instances-0-gw5/instance/configs/config.d Setup database dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node3/database Setup database dir /ClickHouse/tests/integration/test_partition/_instances-0-gw5/instance/database Setup logs dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node3/logs Setup logs dir /ClickHouse/tests/integration/test_partition/_instances-0-gw5/instance/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup directory for instance: node4 Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_partition/_instances-0-gw5/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Create directory for configuration generated in this helper Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_rename_column/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/instant_moves.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/part_log.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/zookeeper_session_timeout.xml', '/ClickHouse/tests/integration/test_rename_column/configs/config.d/storage_configuration.xml'] to /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node4/configs/config.d Setup database dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node4/database Setup logs dir /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node4/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Stdout:1 Volumes pruned: 1 Setup directory for instance: node1 Stdout:1 Create directory for configuration generated in this helper Volumes pruned: 1 Setup directory for instance: node1 Create directory for common tests configuration Stdout:1 Copy common configuration from helpers Volumes pruned: 1 Setup directory for instance: node1 Generate and write macros file Stdout:1 Copy custom test config files [] to /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node1/configs/config.d Volumes pruned: 1 Setup database dir /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node1/database Create directory for configuration generated in this helper Setup directory for instance: node1 Setup logs dir /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node1/logs Stdout:1 Create directory for common tests configuration Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Volumes pruned: 1 Setup directory for instance: node1 Copy common configuration from helpers Setup directory for instance: node2 Create directory for configuration generated in this helper Create directory for common tests configuration Generate and write macros file Copy common configuration from helpers Create directory for configuration generated in this helper Create directory for configuration generated in this helper Create directory for common tests configuration Create directory for common tests configuration Copy custom test config files [] to /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node1/database Copy common configuration from helpers Copy common configuration from helpers Generate and write macros file Create directory for configuration generated in this helper Generate and write macros file Create directory for common tests configuration Generate and write macros file Stdout:1 http://localhost:None "GET /version HTTP/1.1" 200 826 Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/configs/config.d/storage_conf.xml'] to /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/configs/config.d Copy custom test config files ['/ClickHouse/tests/integration/test_part_log_table/configs/config_without_standard_part_log.xml'] to /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node1/configs/config.d Copy common configuration from helpers Volumes pruned: 1 Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_mutations/configs/merge_tree.xml'] to /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node2/configs/config.d Setup directory for instance: instance Generate and write macros file Setup database dir /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node1/database Command:[docker compose --env-file /ClickHouse/tests/integration/test_partition/_instances-0-gw5/.env --project-name roottestpartition-gw5 --file /ClickHouse/tests/integration/test_partition/_instances-0-gw5/instance/docker-compose.yml pull] Setup logs dir /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node1/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup database dir /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/database Setup logs dir /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/logs Setup directory for instance: node2 Copy custom test config files ['/ClickHouse/tests/integration/test_reloading_storage_configuration/configs/logs_config.xml'] to /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node1/configs/config.d Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup database dir /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node2/database Create directory for configuration generated in this helper Setup directory for instance: node2 Create directory for common tests configuration Setup logs dir /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node2/logs Setup logs dir /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node1/logs Create directory for configuration generated in this helper Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Create directory for common tests configuration Create directory for configuration generated in this helper Setup directory for instance: node2 Copy common configuration from helpers Setup directory for instance: node3 Create directory for common tests configuration Copy common configuration from helpers Copy common configuration from helpers Create directory for configuration generated in this helper Create directory for configuration generated in this helper Create directory for common tests configuration Generate and write macros file Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_part_log_table/configs/config_with_standard_part_log.xml'] to /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node2/configs/config.d Copy common configuration from helpers Generate and write macros file Setup database dir /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node2/database Copy custom test config files ['/ClickHouse/tests/integration/test_quota/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_quota/_instances-0-gw1/instance/configs/config.d Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/configs/config.d/storage_conf.xml'] to /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/configs/config.d Generate and write macros file Setup logs dir /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node2/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup database dir /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/database Copy custom test config files [] to /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node2/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node2/database Setup logs dir /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/logs Setup directory for instance: node3 Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_mutations/configs/merge_tree_max_parts.xml'] to /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node3/configs/config.d Stdout:1 Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup logs dir /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node2/logs Volumes pruned: 1 Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup directory for instance: node1 Setup directory for instance: node3 Create directory for configuration generated in this helper Setup directory for instance: node3 Setup database dir /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node3/database Create directory for common tests configuration Create directory for configuration generated in this helper Setup logs dir /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node3/logs Copy common configuration from helpers Create directory for common tests configuration Create directory for configuration generated in this helper Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Create directory for common tests configuration Copy common configuration from helpers Create directory for configuration generated in this helper Generate and write macros file Setup directory for instance: node4 Copy common configuration from helpers Create directory for common tests configuration Copy custom test config files ['/ClickHouse/tests/integration/test_part_log_table/configs/config_with_non_standard_part_log.xml'] to /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node3/configs/config.d Copy common configuration from helpers Generate and write macros file Generate and write macros file Create directory for configuration generated in this helper Setup database dir /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node3/database Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/configs/config.d/storage_conf.xml'] to /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/configs/config.d Create directory for common tests configuration Setup logs dir /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node3/logs Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/configs/limit_replication_config.xml'] to /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node3/configs/config.d Setup database dir /ClickHouse/tests/integration/test_quota/_instances-0-gw1/instance/database Generate and write macros file Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup database dir /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/database Copy common configuration from helpers Setup database dir /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node1/database Setup logs dir /ClickHouse/tests/integration/test_quota/_instances-0-gw1/instance/logs Setup logs dir /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/logs Setup directory for instance: node4 Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup logs dir /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node1/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/configs/encryption_codec.xml'] to /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node1/configs/config.d Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper3/coordination', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/minio/certs/public.crt', 'RESOLVER_LOGS': '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/resolver', 'RESOLVER_LOGS_FS': 'bind'} stored in /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Generate and write macros file Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_quota/_instances-0-gw1/.env Setup database dir /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node3/database Create directory for configuration generated in this helper Setup directory for instance: node2 Create directory for common tests configuration Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_mutations/configs/merge_tree_max_parts.xml'] to /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node4/configs/config.d Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] Setup logs dir /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node3/logs Copy common configuration from helpers No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Create directory for configuration generated in this helper Setup database dir /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node4/database No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] Setup logs dir /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node4/logs Create directory for common tests configuration No config file found Generate and write macros file Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/.env Copy common configuration from helpers Setup database dir /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node1/database Setup directory for instance: node5 Copy custom test config files ['/ClickHouse/tests/integration/test_part_log_table/configs/config_disk_name_test.xml'] to /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node4/configs/config.d Setup logs dir /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node1/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Generate and write macros file Stdout:Total reclaimed space: 0B Create directory for configuration generated in this helper Setup directory for instance: node2 Create directory for common tests configuration Images pruned Trying to prune unused volumes... Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] Copy custom test config files ['/ClickHouse/tests/integration/test_reloading_storage_configuration/configs/logs_config.xml'] to /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node2/configs/config.d Command:[docker volume ls | wc -l] Copy common configuration from helpers No config file found Setup database dir /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node2/database Setup database dir /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node4/database Create directory for configuration generated in this helper Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] Setup logs dir /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node2/logs Create directory for common tests configuration Generate and write macros file No config file found Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Setup logs dir /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node4/logs Copy common configuration from helpers Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/.env Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_mutations/configs/merge_tree_max_parts.xml'] to /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node5/configs/config.d Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup database dir /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node5/database Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/.env Setup logs dir /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node5/logs Generate and write macros file Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/.env No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/configs/encryption_codec.xml'] to /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node2/configs/config.d Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] Setup database dir /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node2/database No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] Setup logs dir /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node2/logs No config file found Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] http://localhost:None "GET /version HTTP/1.1" 200 826 Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] Command:[docker compose --env-file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/.env --project-name roottestrenamecolumn-gw3 --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node4/docker-compose.yml pull] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env --project-name roottestreplicatedmergetrees3zerocopy-gw8 --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/docker-compose.yml pull] Command:[docker compose --env-file /ClickHouse/tests/integration/test_quota/_instances-0-gw1/.env --project-name roottestquota-gw1 --file /ClickHouse/tests/integration/test_quota/_instances-0-gw1/instance/docker-compose.yml pull] http://localhost:None "GET /version HTTP/1.1" 200 826 http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/.env --project-name roottestreplicatedmutations-gw9 --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node5/docker-compose.yml pull] http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/.env --project-name roottestreplicatedfetchesbandwidth-gw6 --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node3/docker-compose.yml pull] Command:[docker compose --env-file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/.env --project-name roottestpartlogtable-gw4 --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node4/docker-compose.yml pull] http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/.env --project-name roottestreloadingstorageconfiguration-gw2 --file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node2/docker-compose.yml pull] http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/.env --project-name roottestreplicatedmergetreeencryptioncodec-gw7 --file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node2/docker-compose.yml pull] Stdout:1 Volumes pruned: 1 Setup directory for instance: instance Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/configs/log_conf.xml'] to /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/configs/config.d Setup database dir /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/database Setup logs dir /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'POSTGRES_PORT': '5432', 'POSTGRES_DIR': '/ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/postgres/postgres1', 'POSTGRES_LOGS_FS': 'bind'} stored in /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/.env --project-name roottestpostgresqlreplicadatabaseengine1-gw0 --file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_postgres.yml pull] Stderr: node2 Skipped - Image is already being pulled by zoo2 Stderr: node3 Skipped - Image is already being pulled by zoo2 Stderr: node4 Skipped - Image is already being pulled by zoo2 Stderr: node5 Skipped - Image is already being pulled by zoo2 Stderr: node1 Skipped - Image is already being pulled by zoo2 Stderr: zoo3 Skipped - Image is already being pulled by zoo2 Stderr: zoo1 Skipped - Image is already being pulled by zoo2 Stderr: zoo2 Pulling Stderr: zoo2 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper1/log', '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper1/config', '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper1/coordination', '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper2/log', '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper2/config', '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper2/coordination', '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper3/log', '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper3/config', '/ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/keeper3/coordination'] Command:[docker compose --project-name roottestreplicatedmutations-gw9 --env-file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stderr: node2 Skipped - Image is already being pulled by node1 Stderr: node3 Skipped - Image is already being pulled by node1 Stderr: node4 Skipped - Image is already being pulled by node1 Stderr: node1 Pulling Stderr: node1 Pulled ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/.env --project-name roottestpartlogtable-gw4 --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node4/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/.env --project-name roottestpartlogtable-gw4 --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node4/docker-compose.yml up -d --no-recreate] Stderr: zoo3 Skipped - Image is already being pulled by zoo2 Stderr: zoo1 Skipped - Image is already being pulled by zoo2 Stderr: node2 Skipped - Image is already being pulled by zoo2 Stderr: node1 Skipped - Image is already being pulled by zoo2 Stderr: zoo2 Pulling Stderr: zoo2 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper1/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper1/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper1/coordination', '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper2/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper2/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper2/coordination', '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper3/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper3/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/keeper3/coordination'] Command:[docker compose --project-name roottestreplicatedmergetreeencryptioncodec-gw7 --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stderr: node4 Skipped - Image is already being pulled by node3 Stderr: node1 Skipped - Image is already being pulled by node3 Stderr: zoo1 Skipped - Image is already being pulled by node3 Stderr: zoo2 Skipped - Image is already being pulled by node3 Stderr: zoo3 Skipped - Image is already being pulled by node3 Stderr: node2 Skipped - Image is already being pulled by node3 Stderr: node3 Pulling Stderr: node3 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper1/log', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper1/config', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper1/coordination', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper2/log', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper2/config', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper2/coordination', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper3/log', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper3/config', '/ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/keeper3/coordination'] Command:[docker compose --project-name roottestrenamecolumn-gw3 --env-file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stderr: instance Pulling Stderr: instance Pulled ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_quota/_instances-0-gw1/.env --project-name roottestquota-gw1 --file /ClickHouse/tests/integration/test_quota/_instances-0-gw1/instance/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_quota/_instances-0-gw1/.env --project-name roottestquota-gw1 --file /ClickHouse/tests/integration/test_quota/_instances-0-gw1/instance/docker-compose.yml up -d --no-recreate] Stderr: zoo1 Skipped - Image is already being pulled by node1 Stderr: zoo2 Skipped - Image is already being pulled by node1 Stderr: zoo3 Skipped - Image is already being pulled by node1 Stderr: node2 Skipped - Image is already being pulled by node1 Stderr: node3 Skipped - Image is already being pulled by node1 Stderr: node1 Pulling Stderr: node1 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper1/log', '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper1/config', '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper1/coordination', '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper2/log', '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper2/config', '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper2/coordination', '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper3/log', '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper3/config', '/ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/keeper3/coordination'] Command:[docker compose --project-name roottestreplicatedfetchesbandwidth-gw6 --env-file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stderr: instance Pulling Stderr: instance Pulled ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_partition/_instances-0-gw5/.env --project-name roottestpartition-gw5 --file /ClickHouse/tests/integration/test_partition/_instances-0-gw5/instance/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_partition/_instances-0-gw5/.env --project-name roottestpartition-gw5 --file /ClickHouse/tests/integration/test_partition/_instances-0-gw5/instance/docker-compose.yml up -d --no-recreate] Stderr: zoo1 Skipped - Image is already being pulled by zoo3 Stderr: node2 Skipped - Image is already being pulled by zoo3 Stderr: node1 Skipped - Image is already being pulled by zoo3 Stderr: zoo2 Skipped - Image is already being pulled by zoo3 Stderr: zoo3 Pulling Stderr: zoo3 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper1/log', '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper1/config', '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper1/coordination', '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper2/log', '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper2/config', '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper2/coordination', '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper3/log', '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper3/config', '/ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/keeper3/coordination'] Command:[docker compose --project-name roottestreloadingstorageconfiguration-gw2 --env-file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stderr:time="2025-04-02T03:31:35Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreplicatedmutations-gw9_default Creating Stderr: Network roottestreplicatedmutations-gw9_default Created Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Creating Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Creating Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Creating Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Created Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Created Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Created Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Starting Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Starting Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Starting Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Started Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Started Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Started Stderr:time="2025-04-02T03:31:36Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:31:36Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.1.3, port:2181, use_ssl:False Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Stderr: zoo2 Skipped - Image is already being pulled by zoo3 Stderr: node3 Skipped - Image is already being pulled by zoo3 Stderr: proxy2 Skipped - Image is already being pulled by proxy1 Stderr: node2 Skipped - Image is already being pulled by zoo3 Stderr: node1 Skipped - Image is already being pulled by zoo3 Stderr: zoo1 Skipped - Image is already being pulled by zoo3 Stderr: minio1 Pulling Stderr: zoo3 Pulling Stderr: proxy1 Pulling Stderr: resolver Pulling Stderr: resolver Pulled Stderr: proxy1 Pulled Stderr: zoo3 Pulled Stderr: minio1 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper1/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper1/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper1/coordination', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper2/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper2/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper2/coordination', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper3/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper3/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/keeper3/coordination'] Command:[docker compose --project-name roottestreplicatedmergetrees3zerocopy-gw8 --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stderr: Network roottestpartlogtable-gw4_default Creating Stderr: Network roottestpartlogtable-gw4_default Created Stderr: Container roottestpartlogtable-gw4-node4-1 Creating Stderr: Container roottestpartlogtable-gw4-node3-1 Creating Stderr: Container roottestpartlogtable-gw4-node1-1 Creating Stderr: Container roottestpartlogtable-gw4-node2-1 Creating Stderr: Container roottestpartlogtable-gw4-node1-1 Created Stderr: Container roottestpartlogtable-gw4-node2-1 Created Stderr: Container roottestpartlogtable-gw4-node3-1 Created Stderr: Container roottestpartlogtable-gw4-node4-1 Created Stderr: Container roottestpartlogtable-gw4-node4-1 Starting Stderr: Container roottestpartlogtable-gw4-node1-1 Starting Stderr: Container roottestpartlogtable-gw4-node2-1 Starting Stderr: Container roottestpartlogtable-gw4-node3-1 Starting Stderr: Container roottestpartlogtable-gw4-node4-1 Started Stderr: Container roottestpartlogtable-gw4-node1-1 Started Stderr: Container roottestpartlogtable-gw4-node3-1 Started Stderr: Container roottestpartlogtable-gw4-node2-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.2.2... http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/e46cdc2355ea07a1c571cd4f5071b2217a182e51adb82668babd87846d17fa54/json HTTP/1.1" 200 None Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/e46cdc2355ea07a1c571cd4f5071b2217a182e51adb82668babd87846d17fa54/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/e46cdc2355ea07a1c571cd4f5071b2217a182e51adb82668babd87846d17fa54/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/e46cdc2355ea07a1c571cd4f5071b2217a182e51adb82668babd87846d17fa54/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.2.5... http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/c8fb0a6f9f242d1f42dcb745156179c0c91d2f40337d855345cff4347451f92b/json HTTP/1.1" 200 None Stderr:time="2025-04-02T03:31:35Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreplicatedmergetreeencryptioncodec-gw7_default Creating Stderr: Network roottestreplicatedmergetreeencryptioncodec-gw7_default Created Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Creating Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Creating Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Creating Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Created Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Created Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Created Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Starting Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Starting Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Starting Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Started Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Started Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Started Stderr:time="2025-04-02T03:31:36Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:31:36Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.3.4, port:2181, use_ssl:False Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/c8fb0a6f9f242d1f42dcb745156179c0c91d2f40337d855345cff4347451f92b/json HTTP/1.1" 200 None ClickHouse node2 started get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node3-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node3-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node3, ip: 172.16.2.3... http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node3-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/f8ccf619b592c6d3dc8cbfe4c6e24263b242634a70d406af88e9bc88a72702be/json HTTP/1.1" 200 None ClickHouse node3 started get_instance_ip instance_name=node4 http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node4-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node4 http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node4-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node4, ip: 172.16.2.4... http://localhost:None "GET /v1.46/containers/roottestpartlogtable-gw4-node4-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/0b128e11b9f2ce7278525b9a1119e95a1f4d8fdafcd0d58e8bbc4ea94167c633/json HTTP/1.1" 200 None ClickHouse node4 started Executing query CREATE TABLE test_table1(word String, value UInt64) ENGINE = MergeTree() ORDER BY word SETTINGS storage_policy = 'test1' on node4 Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Stderr:time="2025-04-02T03:31:35Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestrenamecolumn-gw3_default Creating Stderr: Network roottestrenamecolumn-gw3_default Created Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Creating Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Creating Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Creating Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Created Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Created Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Created Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Starting Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Starting Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Starting Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Started Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Started Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Started Stderr:time="2025-04-02T03:31:37Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:31:37Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 Stderr: Network roottestquota-gw1_default Creating http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-zoo1-1/json HTTP/1.1" 200 None Stderr: Network roottestquota-gw1_default Created Stderr: Container roottestquota-gw1-instance-1 Creating get_kazoo_client: zoo1, ip:172.16.4.3, port:2181, use_ssl:False Stderr: Container roottestquota-gw1-instance-1 Created Stderr: Container roottestquota-gw1-instance-1 Starting Stderr: Container roottestquota-gw1-instance-1 Started ClickHouse instance created get_instance_ip instance_name=instance Connecting to 172.16.4.3(172.16.4.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/roottestquota-gw1-instance-1/json HTTP/1.1" 200 None get_instance_ip instance_name=instance http://localhost:None "GET /v1.46/containers/roottestquota-gw1-instance-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in instance, ip: 172.16.5.2... http://localhost:None "GET /v1.46/containers/roottestquota-gw1-instance-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query INSERT INTO test_table1(*) VALUES ('test1', 2) on node4 Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.4.3(172.16.4.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query CREATE TABLE test_table2(word String, value UInt64) ENGINE = MergeTree() ORDER BY word SETTINGS storage_policy = 'test2' on node4 http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query INSERT INTO test_table2(*) VALUES ('test2', 3) on node4 Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.4.3(172.16.4.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Connecting to 172.16.4.3(172.16.4.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Stderr:time="2025-04-02T03:31:35Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreplicatedfetchesbandwidth-gw6_default Creating Stderr: Network roottestreplicatedfetchesbandwidth-gw6_default Created Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Creating Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Creating Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Creating Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Created Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Created Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Created Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Starting Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Starting Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Starting Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Started Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Started Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Started Stderr:time="2025-04-02T03:31:37Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:31:37Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 Stderr: Network roottestpartition-gw5_default Creating Stderr: Network roottestpartition-gw5_default Created Stderr: Container roottestpartition-gw5-instance-1 Creating Stderr: Container roottestpartition-gw5-instance-1 Created Stderr: Container roottestpartition-gw5-instance-1 Starting Stderr: Container roottestpartition-gw5-instance-1 Started ClickHouse instance created get_instance_ip instance_name=instance http://localhost:None "GET /v1.46/containers/roottestpartition-gw5-instance-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-zoo1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=instance get_kazoo_client: zoo1, ip:172.16.6.2, port:2181, use_ssl:False http://localhost:None "GET /v1.46/containers/roottestpartition-gw5-instance-1/json HTTP/1.1" 200 None Connecting to 172.16.6.2(172.16.6.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Waiting for ClickHouse start in instance, ip: 172.16.7.2... http://localhost:None "GET /v1.46/containers/roottestpartition-gw5-instance-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/e13750de7199458ef980997499e939823d4757128213cd327d9e4f2ef6b092dd/json HTTP/1.1" 200 None Executing query SYSTEM FLUSH LOGS on node4 http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Connecting to 172.16.6.2(172.16.6.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/e13750de7199458ef980997499e939823d4757128213cd327d9e4f2ef6b092dd/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query SELECT DISTINCT disk_name FROM system.part_log ORDER by disk_name on node4 http://localhost:None "GET /v1.46/containers/e13750de7199458ef980997499e939823d4757128213cd327d9e4f2ef6b092dd/json HTTP/1.1" 200 None Connecting to 172.16.6.2(172.16.6.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/e13750de7199458ef980997499e939823d4757128213cd327d9e4f2ef6b092dd/json HTTP/1.1" 200 None Stderr:time="2025-04-02T03:31:36Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreloadingstorageconfiguration-gw2_default Creating Stderr: Network roottestreloadingstorageconfiguration-gw2_default Created Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Creating Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Creating Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Creating Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Created Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Created Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Created Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Starting Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Starting Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Starting Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Started Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Started Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Started Stderr:time="2025-04-02T03:31:37Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:31:37Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreloadingstorageconfiguration-gw2-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.8.2, port:2181, use_ssl:False Connecting to 172.16.8.2(172.16.8.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused [gw4] PASSED test_part_log_table/test.py::test_config_disk_name_test test_part_log_table/test.py::test_config_with_non_standard_part_log Executing query CREATE TABLE test_table(word String, value UInt64) ENGINE=MergeTree() Order by value on node3 http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/e13750de7199458ef980997499e939823d4757128213cd327d9e4f2ef6b092dd/json HTTP/1.1" 200 None ClickHouse instance started Executing query CREATE DATABASE test ENGINE = Ordinary on instance Connecting to 172.16.8.2(172.16.8.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.4.3(172.16.4.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Connecting to 172.16.6.2(172.16.6.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query INSERT INTO test_table VALUES ('name', 1) on node3 Executing query SYSTEM STOP MERGES on instance Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.8.2(172.16.8.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Stderr:time="2025-04-02T03:31:36Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreplicatedmergetrees3zerocopy-gw8_default Creating Stderr: Network roottestreplicatedmergetrees3zerocopy-gw8_default Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Creating Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Creating Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Creating Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Starting Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Starting Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Starting Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Started Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Started Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Started Stderr:time="2025-04-02T03:31:38Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:31:38Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.9.4, port:2181, use_ssl:False Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query DROP TABLE IF EXISTS test.attach_partition on instance Executing query SYSTEM FLUSH LOGS on node3 Executing query CREATE TABLE test.attach_partition (n UInt64) ENGINE = MergeTree() PARTITION BY intDiv(n, 8) ORDER BY n SETTINGS compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1, old_parts_lifetime=0 on instance http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query INSERT INTO test.attach_partition SELECT number FROM system.numbers WHERE number % 2 = 0 LIMIT 8 on instance http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query SELECT * FROM system.own_part_log on node3 Executing query INSERT INTO test.attach_partition SELECT number FROM system.numbers WHERE number % 2 = 1 LIMIT 8 on instance http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query ALTER TABLE test.attach_partition DETACH PARTITION 0 on instance [gw4] PASSED test_part_log_table/test.py::test_config_with_non_standard_part_log test_part_log_table/test.py::test_config_with_standard_part_log Executing query CREATE TABLE test_table(word String, value UInt64) ENGINE=MergeTree() Order by value on node2 Executing query SELECT count() FROM system.parts WHERE active AND rows = 0 AND table = 'attach_partition' AND database = 'test' on instance Connecting to 172.16.8.2(172.16.8.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False Executing query INSERT INTO test_table VALUES ('name', 1) on node2 Connection dropped: socket connection error: Connection refused Executing query SELECT count() FROM system.parts WHERE not active AND table = 'attach_partition' AND database = 'test'; on instance Executing query SYSTEM FLUSH LOGS on node2 Connecting to 172.16.4.3(172.16.4.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/test/attach_partition/detached/0_5_5_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/test/attach_partition/detached/0_5_5_0] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['cp', '-pr', '/var/lib/clickhouse/data/test/attach_partition/detached/0_1_1_0', '/var/lib/clickhouse/data/test/attach_partition/detached/attaching_0_6_6_0'] Command:[docker exec roottestpartition-gw5-instance-1 cp -pr /var/lib/clickhouse/data/test/attach_partition/detached/0_1_1_0 /var/lib/clickhouse/data/test/attach_partition/detached/attaching_0_6_6_0] http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['cp', '-pr', '/var/lib/clickhouse/data/test/attach_partition/detached/0_3_3_0', '/var/lib/clickhouse/data/test/attach_partition/detached/deleting_0_7_7_0'] Command:[docker exec roottestpartition-gw5-instance-1 cp -pr /var/lib/clickhouse/data/test/attach_partition/detached/0_3_3_0 /var/lib/clickhouse/data/test/attach_partition/detached/deleting_0_7_7_0] Connecting to 172.16.6.2(172.16.6.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT * FROM system.part_log on node2 http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query SElECT name FROM system.parts WHERE table='attach_partition' AND database='test' AND active ORDER BY name on instance Executing query SELECT name FROM system.detached_parts WHERE table='attach_partition' AND database='test' ORDER BY name on instance [gw4] PASSED test_part_log_table/test.py::test_config_with_standard_part_log test_part_log_table/test.py::test_config_without_part_log Executing query SELECT * FROM system.part_log on node1 http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query CREATE TABLE test_table(word String, value UInt64) ENGINE=MergeTree() ORDER BY value on node1 Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['rm', '-r', '/var/lib/clickhouse/data/test/attach_partition/detached/0_5_5_0'] Command:[docker exec roottestpartition-gw5-instance-1 rm -r /var/lib/clickhouse/data/test/attach_partition/detached/0_5_5_0] http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query SELECT * FROM system.part_log on node1 Executing query ALTER TABLE test.attach_partition ATTACH PARTITION 0 on instance Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query INSERT INTO test_table VALUES ('name', 1) on node1 Executing query SElECT name FROM system.parts WHERE table='attach_partition' AND database='test' ORDER BY name on instance http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query SYSTEM FLUSH LOGS on node1 Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SElECT count(), sum(n) FROM test.attach_partition on instance http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query SELECT name FROM system.detached_parts WHERE table='attach_partition' AND database='test' ORDER BY name on instance Executing query DROP TABLE test.attach_partition on instance [gw5] PASSED test_partition/test.py::test_attach_check_all_parts Executing query SELECT * FROM system.part_log on node1 http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query SYSTEM START MERGES on instance [gw4] PASSED test_part_log_table/test.py::test_config_without_part_log test_part_log_table/test.py::test_part_log_contains_partition Executing query CREATE TABLE test_partition_table (date Date, word String, value UInt64) ENGINE=MergeTree() PARTITION BY toYYYYMM(date) Order by value on node2 Connecting to 172.16.8.2(172.16.8.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None test_partition/test.py::test_cannot_attach_active_part Executing query DROP TABLE IF EXISTS test.attach_active on instance Executing query INSERT INTO test_partition_table VALUES ('2023-06-20', 'a', 10), ('2023-06-21', 'b', 11),('2023-05-20', 'cc', 14),('2023-05-21', 'd1', 15); on node2 Executing query CREATE TABLE test.attach_active (n UInt64) ENGINE = MergeTree() PARTITION BY intDiv(n, 4) ORDER BY n SETTINGS compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1 on instance Executing query SYSTEM FLUSH LOGS on node2 http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None Executing query INSERT INTO test.attach_active SELECT number FROM system.numbers LIMIT 16 on instance Executing query SELECT partition from system.part_log where table = 'test_partition_table' on node2 http://localhost:None "GET /v1.46/containers/03af55c6ff7c05cb5632d96b2ddcea7784c363fa7303338f8d7d9588360cef4b/json HTTP/1.1" 200 None ClickHouse instance started Executing query DROP TABLE IF EXISTS test_table on instance Executing query SElECT name FROM system.parts WHERE table='attach_active' AND database='test' ORDER BY name on instance Command:[docker compose --env-file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/.env --project-name roottestpartlogtable-gw4 --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node4/docker-compose.yml stop --timeout 20] [gw4] PASSED test_part_log_table/test.py::test_part_log_contains_partition Executing query CREATE TABLE test_table(x UInt32) ENGINE = MergeTree ORDER BY tuple() on instance Executing query SElECT count(), sum(n) FROM test.attach_active on instance Executing query INSERT INTO test_table SELECT number FROM numbers(50) on instance Connecting to 172.16.4.3(172.16.4.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query DROP TABLE test.attach_active on instance [gw5] PASSED test_partition/test.py::test_cannot_attach_active_part Executing query DROP QUOTA IF EXISTS qA, qB on instance test_partition/test.py::test_detached_part_dir_exists Executing query drop table if exists detached_part_dir_exists sync on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.4.4, port:2181, use_ssl:False Connecting to 172.16.4.4(172.16.4.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SYSTEM RELOAD CONFIG on instance Executing query create table detached_part_dir_exists (n int) engine=MergeTree order by n SETTINGS compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1, old_parts_lifetime=0 on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED Executing query insert into detached_part_dir_exists select 1 on instance get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.4.2, port:2181, use_ssl:False Connecting to 172.16.4.2(172.16.4.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query alter table detached_part_dir_exists detach partition id 'all' on instance Executing query SYSTEM RELOAD CONFIG on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/.env --project-name roottestrenamecolumn-gw3 --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node4/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/.env --project-name roottestrenamecolumn-gw3 --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node4/docker-compose.yml up -d --no-recreate] Executing query SELECT count() FROM system.parts WHERE active AND rows = 0 AND table = 'detached_part_dir_exists' AND database = 'default' on instance Connecting to 172.16.6.2(172.16.6.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query SELECT * FROM system.quotas ORDER BY name on instance Executing query SELECT count() FROM system.parts WHERE not active AND table = 'detached_part_dir_exists' AND database = 'default'; on instance Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.6.4, port:2181, use_ssl:False Connecting to 172.16.6.4(172.16.6.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query detach table detached_part_dir_exists on instance Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Executing query attach table detached_part_dir_exists on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.6.3, port:2181, use_ssl:False Connecting to 172.16.6.3(172.16.6.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxkdXJhdGlvbj4zMTU1Njk1MjwvZHVyYXRpb24+CiAgICAgICAgICAgICAgICA8cXVlcmllcz4xMDAwPC9xdWVyaWVzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CgogICAgICAgICAgICA8aW50ZXJ2YWwyPgogICAgICAgICAgICAgICAgPHJhbmRvbWl6ZT50cnVlPC9yYW5kb21pemU+CiAgICAgICAgICAgICAgICA8ZHVyYXRpb24+NjMxMTM5MDQ8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPHJlYWRfYnl0ZXM+MjAwMDA8L3JlYWRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8cmVzdWx0X2J5dGVzPjMwMDAwPC9yZXN1bHRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8ZXhlY3V0aW9uX3RpbWU+MTIwPC9leGVjdXRpb25fdGltZT4KICAgICAgICAgICAgPC9pbnRlcnZhbDI+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxkdXJhdGlvbj4zMTU1Njk1MjwvZHVyYXRpb24+CiAgICAgICAgICAgICAgICA8cXVlcmllcz4xMDAwPC9xdWVyaWVzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CgogICAgICAgICAgICA8aW50ZXJ2YWwyPgogICAgICAgICAgICAgICAgPHJhbmRvbWl6ZT50cnVlPC9yYW5kb21pemU+CiAgICAgICAgICAgICAgICA8ZHVyYXRpb24+NjMxMTM5MDQ8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPHJlYWRfYnl0ZXM+MjAwMDA8L3JlYWRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8cmVzdWx0X2J5dGVzPjMwMDAwPC9yZXN1bHRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8ZXhlY3V0aW9uX3RpbWU+MTIwPC9leGVjdXRpb25fdGltZT4KICAgICAgICAgICAgPC9pbnRlcnZhbDI+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query insert into detached_part_dir_exists select 1 on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') Executing query SYSTEM RELOAD CONFIG on instance ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/.env --project-name roottestreplicatedfetchesbandwidth-gw6 --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node3/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/.env --project-name roottestreplicatedfetchesbandwidth-gw6 --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node3/docker-compose.yml up -d --no-recreate] Executing query insert into detached_part_dir_exists select 1 on instance Executing query SELECT * FROM system.quotas ORDER BY name on instance Executing query select name from system.parts where table='detached_part_dir_exists' and active order by name on instance Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Connecting to 172.16.8.2(172.16.8.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Sending request(xid=2): Close() Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.3.2, port:2181, use_ssl:False Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreloadingstorageconfiguration-gw2-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.8.4, port:2181, use_ssl:False Connecting to 172.16.8.4(172.16.8.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-zoo2-1/json HTTP/1.1" 200 None Received response(xid=1): ['keeper'] get_kazoo_client: zoo2, ip:172.16.1.2, port:2181, use_ssl:False Sending request(xid=2): Close() Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.3.3, port:2181, use_ssl:False Connecting to 172.16.3.3(172.16.3.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 Failed connecting to Zookeeper within the connection retry policy. http://localhost:None "GET /v1.46/containers/roottestreloadingstorageconfiguration-gw2-zoo3-1/json HTTP/1.1" 200 None Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 get_kazoo_client: zoo3, ip:172.16.8.3, port:2181, use_ssl:False Connecting to 172.16.8.3(172.16.8.3):2181, use_ssl: False Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/.env --project-name roottestreplicatedmergetreeencryptioncodec-gw7 --file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node2/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/.env --project-name roottestreplicatedmergetreeencryptioncodec-gw7 --file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node2/docker-compose.yml up -d --no-recreate] http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.1.4, port:2181, use_ssl:False Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Zookeeper connection established, state: CONNECTED Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Running Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Running Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Running Stderr: Container roottestrenamecolumn-gw3-node4-1 Creating Stderr: Container roottestrenamecolumn-gw3-node2-1 Creating Stderr: Container roottestrenamecolumn-gw3-node1-1 Creating Stderr: Container roottestrenamecolumn-gw3-node3-1 Creating Stderr: Container roottestrenamecolumn-gw3-node3-1 Created Stderr: Container roottestrenamecolumn-gw3-node2-1 Created Stderr: Container roottestrenamecolumn-gw3-node4-1 Created Stderr: Container roottestrenamecolumn-gw3-node1-1 Created Stderr: Container roottestrenamecolumn-gw3-node3-1 Starting Stderr: Container roottestrenamecolumn-gw3-node4-1 Starting Stderr: Container roottestrenamecolumn-gw3-node1-1 Starting Stderr: Container roottestrenamecolumn-gw3-node2-1 Starting Stderr: Container roottestrenamecolumn-gw3-node3-1 Started Stderr: Container roottestrenamecolumn-gw3-node2-1 Started Stderr: Container roottestrenamecolumn-gw3-node4-1 Started Stderr: Container roottestrenamecolumn-gw3-node1-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.4.8... http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/b15139c1a083916f24843a20389487b6021bf2c91f2f9295dba163d9b4475999/json HTTP/1.1" 200 None Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir /var/lib/clickhouse/data/default/detached_part_dir_exists/detached/all_2_2_0'] Command:[docker exec --privileged roottestpartition-gw5-instance-1 bash -c mkdir /var/lib/clickhouse/data/default/detached_part_dir_exists/detached/all_2_2_0] Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/.env --project-name roottestreplicatedmutations-gw9 --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node5/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/.env --project-name roottestreplicatedmutations-gw9 --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node5/docker-compose.yml up -d --no-recreate] Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/.env --project-name roottestreloadingstorageconfiguration-gw2 --file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node2/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/.env --project-name roottestreloadingstorageconfiguration-gw2 --file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node2/docker-compose.yml up -d --no-recreate] http://localhost:None "GET /v1.46/containers/b15139c1a083916f24843a20389487b6021bf2c91f2f9295dba163d9b4475999/json HTTP/1.1" 200 None run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'touch /var/lib/clickhouse/data/default/detached_part_dir_exists/detached/all_2_2_0/file'] Command:[docker exec --privileged roottestpartition-gw5-instance-1 bash -c touch /var/lib/clickhouse/data/default/detached_part_dir_exists/detached/all_2_2_0/file] Executing query SELECT * from test_table on instance Executing query alter table detached_part_dir_exists detach partition id 'all' on instance Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance http://localhost:None "GET /v1.46/containers/b15139c1a083916f24843a20389487b6021bf2c91f2f9295dba163d9b4475999/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/b15139c1a083916f24843a20389487b6021bf2c91f2f9295dba163d9b4475999/json HTTP/1.1" 200 None Connecting to 172.16.9.4(172.16.9.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Running Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Running Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Running Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node2-1 Creating Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node3-1 Creating Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node1-1 Creating Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node3-1 Created Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node1-1 Created Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node2-1 Created Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node2-1 Starting Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node3-1 Starting Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node1-1 Starting Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node1-1 Started Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node3-1 Started Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node2-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.6.7... http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/5e179a79622deb5fbb0b2069c0e4d5c48bc31912b6373f393a8d6fdd3ba8ea83/json HTTP/1.1" 200 None Executing query select name from system.detached_parts where table='detached_part_dir_exists' order by name on instance http://localhost:None "GET /v1.46/containers/b15139c1a083916f24843a20389487b6021bf2c91f2f9295dba163d9b4475999/json HTTP/1.1" 200 None run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.9.3, port:2181, use_ssl:False Connecting to 172.16.9.3(172.16.9.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Executing query SYSTEM RELOAD CONFIG on instance Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost http://localhost:None "GET /v1.46/containers/5e179a79622deb5fbb0b2069c0e4d5c48bc31912b6373f393a8d6fdd3ba8ea83/json HTTP/1.1" 200 None Stderr: instance Pulling Stderr: postgres1 Pulling Stderr: 6e909acdb790 Pulling fs layer Stderr: fec99121872b Pulling fs layer Stderr: 133acbc970df Pulling fs layer Stderr: e02d97322fc6 Pulling fs layer Stderr: db9643c6baf3 Pulling fs layer Stderr: 9bcedd9434e7 Pulling fs layer Stderr: fc8982ec96d9 Pulling fs layer Stderr: 1824bd6b75d7 Pulling fs layer Stderr: fbad2bf2d5e6 Pulling fs layer Stderr: 221788d72606 Pulling fs layer Stderr: e5f43b682bc0 Pulling fs layer Stderr: e7a2d9e24ab0 Pulling fs layer Stderr: a96cb29b0d13 Pulling fs layer Stderr: 140970538145 Pulling fs layer Stderr: 9bcedd9434e7 Waiting Stderr: 1824bd6b75d7 Waiting Stderr: e02d97322fc6 Waiting Stderr: e7a2d9e24ab0 Waiting Stderr: e5f43b682bc0 Waiting Stderr: 140970538145 Waiting Stderr: db9643c6baf3 Waiting Stderr: fbad2bf2d5e6 Waiting Stderr: 221788d72606 Waiting Stderr: instance Pulled Stderr: 133acbc970df Downloading [> ] 45.64kB/4.534MB Stderr: fec99121872b Downloading [==================================================>] 1.168kB/1.168kB Stderr: fec99121872b Verifying Checksum Stderr: fec99121872b Download complete Stderr: e02d97322fc6 Downloading [=> ] 42.74kB/1.447MB Stderr: 6e909acdb790 Downloading [> ] 298.5kB/28.2MB Stderr: 133acbc970df Download complete Stderr: e02d97322fc6 Downloading [==================================================>] 1.447MB/1.447MB Stderr: e02d97322fc6 Verifying Checksum Stderr: e02d97322fc6 Download complete Stderr: 9bcedd9434e7 Downloading [> ] 13.78kB/1.196MB Stderr: db9643c6baf3 Downloading [> ] 99.21kB/8.066MB Stderr: 9bcedd9434e7 Verifying Checksum Stderr: 9bcedd9434e7 Download complete Stderr: fc8982ec96d9 Downloading [==================================================>] 116B/116B Stderr: fc8982ec96d9 Verifying Checksum Stderr: fc8982ec96d9 Download complete Stderr: 1824bd6b75d7 Downloading [==================================================>] 3.143kB/3.143kB Stderr: 1824bd6b75d7 Verifying Checksum Stderr: 1824bd6b75d7 Download complete Stderr: fbad2bf2d5e6 Downloading [> ] 544.7kB/112.8MB Stderr: db9643c6baf3 Verifying Checksum Stderr: db9643c6baf3 Download complete Stderr: 221788d72606 Downloading [=================> ] 3.646kB/10.24kB Stderr: 221788d72606 Download complete Stderr: e5f43b682bc0 Downloading [==================================================>] 129B/129B Stderr: e5f43b682bc0 Verifying Checksum Stderr: e5f43b682bc0 Download complete Stderr: e7a2d9e24ab0 Downloading [==================================================>] 167B/167B Stderr: e7a2d9e24ab0 Verifying Checksum Stderr: e7a2d9e24ab0 Download complete Stderr: a96cb29b0d13 Downloading [=================================> ] 3.647kB/5.473kB Stderr: a96cb29b0d13 Downloading [==================================================>] 5.473kB/5.473kB Stderr: a96cb29b0d13 Verifying Checksum Stderr: a96cb29b0d13 Download complete Stderr: 6e909acdb790 Downloading [=====================================> ] 20.94MB/28.2MB Stderr: 140970538145 Downloading [==================================================>] 185B/185B Stderr: 140970538145 Verifying Checksum Stderr: 140970538145 Download complete Stderr: 6e909acdb790 Verifying Checksum Stderr: 6e909acdb790 Download complete Stderr: fbad2bf2d5e6 Downloading [===============> ] 33.95MB/112.8MB Stderr: 6e909acdb790 Extracting [> ] 294.9kB/28.2MB Stderr: fbad2bf2d5e6 Downloading [==============================> ] 69.05MB/112.8MB Stderr: 6e909acdb790 Extracting [=====> ] 3.244MB/28.2MB Stderr: fbad2bf2d5e6 Downloading [===========================================> ] 98.02MB/112.8MB Stderr: 6e909acdb790 Extracting [===========> ] 6.488MB/28.2MB Stderr: fbad2bf2d5e6 Verifying Checksum Stderr: fbad2bf2d5e6 Download complete Stderr: 6e909acdb790 Extracting [================> ] 9.437MB/28.2MB Stderr: 6e909acdb790 Extracting [===================> ] 11.21MB/28.2MB Stderr: 6e909acdb790 Extracting [===========================> ] 15.63MB/28.2MB Stderr: 6e909acdb790 Extracting [========================================> ] 23MB/28.2MB Stderr: 6e909acdb790 Extracting [==========================================> ] 23.89MB/28.2MB Stderr: 6e909acdb790 Extracting [===============================================> ] 26.84MB/28.2MB Stderr: 6e909acdb790 Extracting [================================================> ] 27.43MB/28.2MB Stderr: 6e909acdb790 Extracting [==================================================>] 28.2MB/28.2MB Stderr: 6e909acdb790 Pull complete Stderr: fec99121872b Extracting [==================================================>] 1.168kB/1.168kB Stderr: fec99121872b Extracting [==================================================>] 1.168kB/1.168kB Stderr: fec99121872b Pull complete Stderr: 133acbc970df Extracting [> ] 65.54kB/4.534MB Stderr: 133acbc970df Extracting [=================================================> ] 4.522MB/4.534MB Stderr: 133acbc970df Extracting [==================================================>] 4.534MB/4.534MB Stderr: 133acbc970df Pull complete Stderr: e02d97322fc6 Extracting [=> ] 32.77kB/1.447MB Stderr: e02d97322fc6 Extracting [==================================================>] 1.447MB/1.447MB Stderr: e02d97322fc6 Pull complete Stderr: db9643c6baf3 Extracting [> ] 98.3kB/8.066MB Stderr: db9643c6baf3 Extracting [==================> ] 3.047MB/8.066MB Stderr: db9643c6baf3 Extracting [=============================> ] 4.719MB/8.066MB Stderr: db9643c6baf3 Extracting [======================================> ] 6.291MB/8.066MB Stderr: db9643c6baf3 Extracting [==================================================>] 8.066MB/8.066MB Stderr: db9643c6baf3 Pull complete Stderr: 9bcedd9434e7 Extracting [=> ] 32.77kB/1.196MB Stderr: 9bcedd9434e7 Extracting [==================================================>] 1.196MB/1.196MB Stderr: 9bcedd9434e7 Extracting [==================================================>] 1.196MB/1.196MB Stderr: 9bcedd9434e7 Pull complete Stderr: fc8982ec96d9 Extracting [==================================================>] 116B/116B Stderr: fc8982ec96d9 Extracting [==================================================>] 116B/116B Stderr: fc8982ec96d9 Pull complete Stderr: 1824bd6b75d7 Extracting [==================================================>] 3.143kB/3.143kB Stderr: 1824bd6b75d7 Extracting [==================================================>] 3.143kB/3.143kB Stderr: 1824bd6b75d7 Pull complete Stderr: fbad2bf2d5e6 Extracting [> ] 557.1kB/112.8MB Stderr: fbad2bf2d5e6 Extracting [===> ] 7.799MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [====> ] 10.03MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [=====> ] 13.37MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [=======> ] 16.71MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [=========> ] 20.61MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [==========> ] 23.95MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [=============> ] 30.64MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [=================> ] 39.55MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [=====================> ] 48.46MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [========================> ] 54.59MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [=========================> ] 58.49MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [===========================> ] 62.95MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [===============================> ] 70.75MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [===================================> ] 80.22MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [=======================================> ] 88.01MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [==========================================> ] 95.81MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [===========================================> ] 97.48MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [============================================> ] 100.8MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [==============================================> ] 104.2MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [===============================================> ] 107MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [================================================> ] 108.6MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [================================================> ] 109.7MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [=================================================> ] 111.4MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [=================================================> ] 112.5MB/112.8MB Stderr: fbad2bf2d5e6 Extracting [==================================================>] 112.8MB/112.8MB Stderr: fbad2bf2d5e6 Pull complete Stderr: 221788d72606 Extracting [==================================================>] 10.24kB/10.24kB Stderr: 221788d72606 Extracting [==================================================>] 10.24kB/10.24kB Stderr: 221788d72606 Pull complete Stderr: e5f43b682bc0 Extracting [==================================================>] 129B/129B Stderr: e5f43b682bc0 Extracting [==================================================>] 129B/129B Stderr: e5f43b682bc0 Pull complete Stderr: e7a2d9e24ab0 Extracting [==================================================>] 167B/167B Stderr: e7a2d9e24ab0 Extracting [==================================================>] 167B/167B Stderr: e7a2d9e24ab0 Pull complete Stderr: a96cb29b0d13 Extracting [==================================================>] 5.473kB/5.473kB Stderr: a96cb29b0d13 Extracting [==================================================>] 5.473kB/5.473kB Stderr: a96cb29b0d13 Pull complete Stderr: 140970538145 Extracting [==================================================>] 185B/185B Stderr: 140970538145 Extracting [==================================================>] 185B/185B Stderr: 140970538145 Pull complete Stderr: postgres1 Pulled Setup Postgres Command:[docker compose --project-name roottestpostgresqlreplicadatabaseengine1-gw0 --env-file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_postgres.yml --verbose up -d] http://localhost:None "GET /v1.46/containers/b15139c1a083916f24843a20389487b6021bf2c91f2f9295dba163d9b4475999/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.4.7... http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/c3b8b674a9316bc14eac8a2e5938063f739de4aaf8b366eef97eabb32f72442a/json HTTP/1.1" 200 None ClickHouse node2 started get_instance_ip instance_name=node3 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node3-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node3-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/5e179a79622deb5fbb0b2069c0e4d5c48bc31912b6373f393a8d6fdd3ba8ea83/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node3, ip: 172.16.4.5... get_kazoo_client: zoo3, ip:172.16.9.2, port:2181, use_ssl:False Connecting to 172.16.9.2(172.16.9.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Executing query SELECT * FROM system.quotas ORDER BY name on instance http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node3-1/json HTTP/1.1" 200 None Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost http://localhost:None "GET /v1.46/containers/80acf1456022962f64a1ca65010c1ce5baad04cfd83399dd59102dea6bb491a3/json HTTP/1.1" 200 None ClickHouse node3 started get_instance_ip instance_name=node4 http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node4-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node4 Executing query drop table detached_part_dir_exists on instance http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node4-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node4, ip: 172.16.4.6... http://localhost:None "GET /v1.46/containers/roottestrenamecolumn-gw3-node4-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/db851d0684ba2feb49995668bdb001ec7f82fd196262ee59e5a095ad27e68ee0/json HTTP/1.1" 200 None ClickHouse node4 started Executing query CREATE TABLE test_rename_distributed_replicated ON CLUSTER test_cluster ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/{shard}/test_rename_distributed_replicated', '{replica}') ORDER BY num PARTITION BY num % 100; on node1 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') Trying to create Minio instance by command docker compose --project-name roottestreplicatedmergetrees3zerocopy-gw8 --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --verbose up -d Command:[docker compose --project-name roottestreplicatedmergetrees3zerocopy-gw8 --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --verbose up -d] http://localhost:None "GET /v1.46/containers/5e179a79622deb5fbb0b2069c0e4d5c48bc31912b6373f393a8d6fdd3ba8ea83/json HTTP/1.1" 200 None [gw5] PASSED test_partition/test.py::test_detached_part_dir_exists test_partition/test.py::test_drop_detached_parts Executing query SYSTEM STOP MERGES on instance Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Executing query CREATE TABLE test_rename_distributed ON CLUSTER test_cluster AS test_rename_distributed_replicated ENGINE = Distributed(test_cluster, default, test_rename_distributed_replicated, rand()) on node1 http://localhost:None "GET /v1.46/containers/5e179a79622deb5fbb0b2069c0e4d5c48bc31912b6373f393a8d6fdd3ba8ea83/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.6.5... http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/0208ec8777d3628ec386579ad585206a8225d28c0218965300b89dfde981fcaa/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/0208ec8777d3628ec386579ad585206a8225d28c0218965300b89dfde981fcaa/json HTTP/1.1" 200 None ClickHouse node2 started get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-node3-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-node3-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node3, ip: 172.16.6.6... http://localhost:None "GET /v1.46/containers/roottestreplicatedfetchesbandwidth-gw6-node3-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/d2013f99655299c27106a233a85d1544e1471bb3477fa732a35b8cb7c7ec763a/json HTTP/1.1" 200 None ClickHouse node3 started Executing query CREATE TABLE limited_fetch_table(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetch_table', '0') ORDER BY tuple() PARTITION BY key SETTINGS max_replicated_fetches_network_bandwidth=10485760 on node1 Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Running Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Running Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Running Stderr: Container roottestreloadingstorageconfiguration-gw2-node1-1 Creating Stderr: Container roottestreloadingstorageconfiguration-gw2-node2-1 Creating Stderr: Container roottestreloadingstorageconfiguration-gw2-node1-1 Created Stderr: Container roottestreloadingstorageconfiguration-gw2-node2-1 Created Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Running Stderr: Container roottestreloadingstorageconfiguration-gw2-node2-1 Starting Stderr: Container roottestreloadingstorageconfiguration-gw2-node1-1 Starting Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Running Stderr: Container roottestreloadingstorageconfiguration-gw2-node1-1 Started Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Running Stderr: Container roottestreloadingstorageconfiguration-gw2-node2-1 Started ClickHouse instance created Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 Creating get_instance_ip instance_name=node1 Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 Creating Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 Created Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 Created Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 Starting Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 Starting Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 Started Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencryptioncodec-gw7-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.8.5... http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencryptioncodec-gw7-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.3.6... http://localhost:None "GET /v1.46/containers/5737c7f01c66045ae30712b0b852893c72a3a6daf533aeb5fe6f8dbf49cafb15/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencryptioncodec-gw7-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2f1f54479600ea5a2af0bbfea20ea0f8e1a9df2c85671ad7d430b021450c0812/json HTTP/1.1" 200 None Executing query DROP TABLE IF EXISTS test.drop_detached on instance Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Stderr: Container roottestreplicatedmutations-gw9-node5-1 Creating Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Running Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Running Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Running Stderr: Container roottestreplicatedmutations-gw9-node2-1 Creating Stderr: Container roottestreplicatedmutations-gw9-node3-1 Creating Stderr: Container roottestreplicatedmutations-gw9-node4-1 Creating Stderr: Container roottestreplicatedmutations-gw9-node1-1 Creating Stderr: Container roottestreplicatedmutations-gw9-node3-1 Created Stderr: Container roottestreplicatedmutations-gw9-node5-1 Created Stderr: Container roottestreplicatedmutations-gw9-node4-1 Created Stderr: Container roottestreplicatedmutations-gw9-node1-1 Created Stderr: Container roottestreplicatedmutations-gw9-node2-1 Created Stderr: Container roottestreplicatedmutations-gw9-node5-1 Starting Stderr: Container roottestreplicatedmutations-gw9-node1-1 Starting Stderr: Container roottestreplicatedmutations-gw9-node4-1 Starting Stderr: Container roottestreplicatedmutations-gw9-node2-1 Starting Stderr: Container roottestreplicatedmutations-gw9-node3-1 Starting Stderr: Container roottestreplicatedmutations-gw9-node4-1 Started Stderr: Container roottestreplicatedmutations-gw9-node3-1 Started Stderr: Container roottestreplicatedmutations-gw9-node5-1 Started Stderr: Container roottestreplicatedmutations-gw9-node2-1 Started Stderr: Container roottestreplicatedmutations-gw9-node1-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node1-1/json HTTP/1.1" 200 None Executing query CREATE TABLE limited_fetch_table(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetch_table', '1') ORDER BY tuple() PARTITION BY key SETTINGS max_replicated_fetches_network_bandwidth=10485760 on node2 Waiting for ClickHouse start in node1, ip: 172.16.1.9... http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/37d9597d5a57dd781968f1faf65f979adba7b334830cad2640bcd25fc45de5cb/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/5737c7f01c66045ae30712b0b852893c72a3a6daf533aeb5fe6f8dbf49cafb15/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2f1f54479600ea5a2af0bbfea20ea0f8e1a9df2c85671ad7d430b021450c0812/json HTTP/1.1" 200 None Executing query CREATE TABLE test.drop_detached (n UInt64) ENGINE = MergeTree() PARTITION BY intDiv(n, 8) ORDER BY n SETTINGS compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1 on instance Executing query SELECT * from test_table on instance Executing query SYSTEM STOP FETCHES limited_fetch_table on node2 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(1000) on node1 http://localhost:None "GET /v1.46/containers/37d9597d5a57dd781968f1faf65f979adba7b334830cad2640bcd25fc45de5cb/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/5737c7f01c66045ae30712b0b852893c72a3a6daf533aeb5fe6f8dbf49cafb15/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2f1f54479600ea5a2af0bbfea20ea0f8e1a9df2c85671ad7d430b021450c0812/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/37d9597d5a57dd781968f1faf65f979adba7b334830cad2640bcd25fc45de5cb/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/5737c7f01c66045ae30712b0b852893c72a3a6daf533aeb5fe6f8dbf49cafb15/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2f1f54479600ea5a2af0bbfea20ea0f8e1a9df2c85671ad7d430b021450c0812/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/37d9597d5a57dd781968f1faf65f979adba7b334830cad2640bcd25fc45de5cb/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/5737c7f01c66045ae30712b0b852893c72a3a6daf533aeb5fe6f8dbf49cafb15/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 Stderr:time="2025-04-02T03:31:41Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestpostgresqlreplicadatabaseengine1-gw0_default Creating Stderr: Network roottestpostgresqlreplicadatabaseengine1-gw0_default Created Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Creating Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Created Executing query INSERT INTO limited_fetch_table SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(300) on node1 Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Starting Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Started Stderr:time="2025-04-02T03:31:42Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:31:42Z" level=debug msg="otel error" error="" http://localhost:None "GET /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=postgres1 Waiting for ClickHouse start in node2, ip: 172.16.8.6... http://localhost:None "GET /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2f1f54479600ea5a2af0bbfea20ea0f8e1a9df2c85671ad7d430b021450c0812/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/4bc95afd17a5fa0cff62a74213d07fcb8f332c4f03dc60868a20d7eebb8d9f55/json HTTP/1.1" 200 None Can't connect to Postgres connection to server at "172.16.10.2", port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections? Executing query INSERT INTO test.drop_detached SELECT number FROM system.numbers WHERE number % 2 = 0 LIMIT 8 on instance Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance http://localhost:None "GET /v1.46/containers/37d9597d5a57dd781968f1faf65f979adba7b334830cad2640bcd25fc45de5cb/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.1.6... http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/1586045e532843e17d8e7f151ba2d9e129c45e2f93d837a663f18efb458a82f8/json HTTP/1.1" 200 None ClickHouse node2 started get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node3-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node3-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node3, ip: 172.16.1.8... http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node3-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/5aef59b48251b10ad6c77233b04a5bf5b81a2f3a3d56ffa8a2b8fc96f5ab5bf6/json HTTP/1.1" 200 None ClickHouse node3 started get_instance_ip instance_name=node4 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node4-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node4 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node4-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node4, ip: 172.16.1.7... http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node4-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/744e47e9015b9737c5217254cbde02e972454cbb06f6f6cdb46fbac494586d6c/json HTTP/1.1" 200 None ClickHouse node4 started get_instance_ip instance_name=node5 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node5-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node5 http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node5-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node5, ip: 172.16.1.5... http://localhost:None "GET /v1.46/containers/roottestreplicatedmutations-gw9-node5-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/c2978d0ed1f98e218f7fb3eec61f3eb00d4f0d89a7d9eb881e2a16b60f72a282/json HTTP/1.1" 200 None ClickHouse node5 started http://localhost:None "GET /v1.46/containers/4bc95afd17a5fa0cff62a74213d07fcb8f332c4f03dc60868a20d7eebb8d9f55/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2f1f54479600ea5a2af0bbfea20ea0f8e1a9df2c85671ad7d430b021450c0812/json HTTP/1.1" 200 None Executing query DROP TABLE IF EXISTS test_mutations SYNC on node1 Executing query INSERT INTO limited_fetch_table SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(300) on node1 Executing query DROP TABLE IF EXISTS test_mutations SYNC on node2 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query INSERT INTO test.drop_detached SELECT number FROM system.numbers WHERE number % 2 = 1 LIMIT 8 on instance http://localhost:None "GET /v1.46/containers/2f1f54479600ea5a2af0bbfea20ea0f8e1a9df2c85671ad7d430b021450c0812/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/4bc95afd17a5fa0cff62a74213d07fcb8f332c4f03dc60868a20d7eebb8d9f55/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 ClickHouse node2 started http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencryptioncodec-gw7-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencryptioncodec-gw7-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.3.5... run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencryptioncodec-gw7-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/428614a204c01e170d94d4e439c69703e8495a8cbd3d47e3c07270858d086962/json HTTP/1.1" 200 None ClickHouse node2 started run container_id:roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleSByZW1vdmU9IjEiLz4KICAgICAgICAgICAgPGtleSBpZD0iMCI+YWFhYWFhYWFhYWFhYWFhYTwva2V5PgogICAgICAgICAgICA8a2V5IGlkPSIxIj5iYmJiYmJiYmJiYmJiYmJiPC9rZXk+CiAgICAgICAgICAgIDxjdXJyZW50X2tleV9pZD4wPC9jdXJyZW50X2tleV9pZD4KICAgICAgICA8L2Flc18xMjhfZ2NtX3Npdj4KICAgIDwvZW5jcnlwdGlvbl9jb2RlY3M+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleSByZW1vdmU9IjEiLz4KICAgICAgICAgICAgPGtleSBpZD0iMCI+YWFhYWFhYWFhYWFhYWFhYTwva2V5PgogICAgICAgICAgICA8a2V5IGlkPSIxIj5iYmJiYmJiYmJiYmJiYmJiPC9rZXk+CiAgICAgICAgICAgIDxjdXJyZW50X2tleV9pZD4wPC9jdXJyZW50X2tleV9pZD4KICAgICAgICA8L2Flc18xMjhfZ2NtX3Npdj4KICAgIDwvZW5jcnlwdGlvbl9jb2RlY3M+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] Executing query DROP TABLE IF EXISTS test_mutations SYNC on node3 Executing query SYSTEM RELOAD CONFIG on instance Executing query ALTER TABLE test.drop_detached DETACH PARTITION 0 on instance Stdout: PID TTY TIME CMD Stdout: 10 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] Executing query SYSTEM RELOAD CONFIG on node1 Executing query DROP TABLE IF EXISTS test_mutations SYNC on node4 Executing query ALTER TABLE test.drop_detached DETACH PARTITION 1 on instance run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query INSERT INTO limited_fetch_table SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(300) on node1 run container_id:roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleSByZW1vdmU9IjEiLz4KICAgICAgICAgICAgPGtleSBpZD0iMCI+YWFhYWFhYWFhYWFhYWFhYTwva2V5PgogICAgICAgICAgICA8a2V5IGlkPSIxIj5iYmJiYmJiYmJiYmJiYmJiPC9rZXk+CiAgICAgICAgICAgIDxjdXJyZW50X2tleV9pZD4xPC9jdXJyZW50X2tleV9pZD4KICAgICAgICA8L2Flc18xMjhfZ2NtX3Npdj4KICAgIDwvZW5jcnlwdGlvbl9jb2RlY3M+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleSByZW1vdmU9IjEiLz4KICAgICAgICAgICAgPGtleSBpZD0iMCI+YWFhYWFhYWFhYWFhYWFhYTwva2V5PgogICAgICAgICAgICA8a2V5IGlkPSIxIj5iYmJiYmJiYmJiYmJiYmJiPC9rZXk+CiAgICAgICAgICAgIDxjdXJyZW50X2tleV9pZD4xPC9jdXJyZW50X2tleV9pZD4KICAgICAgICA8L2Flc18xMjhfZ2NtX3Npdj4KICAgIDwvZW5jcnlwdGlvbl9jb2RlY3M+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] Stdout:10 Executing query SELECT * FROM system.quotas ORDER BY name on instance Executing query DROP TABLE IF EXISTS test_mutations SYNC on node5 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/test/drop_detached/detached/attaching_0_6_6_0'] Executing query SYSTEM RELOAD CONFIG on node2 Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/test/drop_detached/detached/attaching_0_6_6_0] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/test/drop_detached/detached/deleting_0_7_7_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/test/drop_detached/detached/deleting_0_7_7_0] Executing query INSERT INTO limited_fetch_table SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(300) on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/test/drop_detached/detached/any_other_name'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/test/drop_detached/detached/any_other_name] Can't connect to Postgres connection to server at "172.16.10.2", port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections? run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/test/drop_detached/detached/prefix_1_2_2_0_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/test/drop_detached/detached/prefix_1_2_2_0_0] Stderr:time="2025-04-02T03:31:41Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Volume "roottestreplicatedmergetrees3zerocopy-gw8_data1-1" Creating Stderr: Volume "roottestreplicatedmergetrees3zerocopy-gw8_data1-1" Created Stderr:time="2025-04-02T03:31:41Z" level=warning msg="Found orphan containers ([roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up." Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Creating Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Creating Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Creating Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Creating Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Starting Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Starting Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Started Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Started Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Starting Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Starting Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Started Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Started Stderr:time="2025-04-02T03:31:43Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:31:43Z" level=debug msg="otel error" error="" Trying to connect to Minio... get_instance_ip instance_name=minio1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-minio1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=proxy1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1/json HTTP/1.1" 200 None Starting new HTTP connection (1): 172.16.9.7:9001 Incremented Retry for (url='/'): Retry(total=2, connect=None, read=None, redirect=None, status=None) Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / Starting new HTTP connection (2): 172.16.9.7:9001 Executing query DROP TABLE IF EXISTS tbl ON CLUSTER 'cluster' SYNC on node1 Incremented Retry for (url='/'): Retry(total=1, connect=None, read=None, redirect=None, status=None) Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / Starting new HTTP connection (3): 172.16.9.7:9001 Incremented Retry for (url='/'): Retry(total=0, connect=None, read=None, redirect=None, status=None) Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / Starting new HTTP connection (4): 172.16.9.7:9001 Can't connect to Minio: HTTPConnectionPool(host='172.16.9.7', port=9001): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node1 Executing query ALTER TABLE test.drop_detached DROP DETACHED PART '0_1_1_0' on instance Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Executing query INSERT INTO limited_fetch_table SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(300) on node1 Executing query ALTER TABLE test_rename_distributed ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 Can't connect to Postgres connection to server at "172.16.10.2", port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections? Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node2 Executing query SELECT * from test_table on instance Executing query ALTER TABLE test_rename_distributed_replicated ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 Stderr: Container roottestpartlogtable-gw4-node2-1 Stopping Stderr: Container roottestpartlogtable-gw4-node1-1 Stopping Stderr: Container roottestpartlogtable-gw4-node3-1 Stopping Stderr: Container roottestpartlogtable-gw4-node4-1 Stopping Stderr: Container roottestpartlogtable-gw4-node2-1 Stopped Stderr: Container roottestpartlogtable-gw4-node1-1 Stopped Stderr: Container roottestpartlogtable-gw4-node4-1 Stopped Stderr: Container roottestpartlogtable-gw4-node3-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Executing query CREATE TABLE tbl ON CLUSTER 'cluster' ( id Int64, str String Codec(AES_128_GCM_SIV) ) ENGINE=ReplicatedMergeTree('/clickhouse/tables/tbl/', '{replica}') ORDER BY id on node1 Command:[bash -c [ -f /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node4/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node4/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk \'{print $1 " " $2}\' /proc/net/route | grep 00000000 | awk \'{print $1}\''] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk '{print $1 " " $2}' /proc/net/route | grep 00000000 | awk '{print $1}'] Command:[docker compose --env-file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/.env --project-name roottestpartlogtable-gw4 --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_part_log_table/_instances-0-gw4/node4/docker-compose.yml down --volumes] Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node3 Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Stdout:eth0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query ALTER TABLE test.drop_detached DROP DETACHED PART 'any_other_name' on instance Stdout: eth0: 33293 222 0 0 0 0 0 0 51729 219 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node4 Executing query SElECT name FROM system.detached_parts WHERE table='drop_detached' AND database='test' ORDER BY name on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query INSERT INTO tbl VALUES (1, 'str1') on node1 Stdout: eth0: 33293 222 0 0 0 0 0 0 51729 219 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query SYSTEM RELOAD CONFIG on instance Executing query ALTER TABLE test.drop_detached DROP DETACHED PARTITION 1 on instance Stdout:33293 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE MergeTree() ORDER BY x PARTITION BY toYYYYMM(d) on node5 Stdout:51729 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk \'{print $1 " " $2}\' /proc/net/route | grep 00000000 | awk \'{print $1}\''] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk '{print $1 " " $2}' /proc/net/route | grep 00000000 | awk '{print $1}'] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:eth0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query SElECT name FROM system.detached_parts WHERE table='drop_detached' AND database='test' ORDER BY name on instance Executing query INSERT INTO tbl VALUES (1, 'str1') on node2 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout: eth0: 24634 171 0 0 0 0 0 0 24723 167 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query SELECT * FROM system.quotas ORDER BY name on instance Stdout: eth0: 24634 171 0 0 0 0 0 0 24723 167 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 Executing query DROP TABLE test.drop_detached on instance Executing query INSERT INTO tbl VALUES (2, 'str2') on node2 [gw5] PASSED test_partition/test.py::test_drop_detached_parts http://localhost:None "POST /v1.46/exec/469573da0fd3b06314dcc83a54493c66b6d169706d8f72be9bc563eb360f9cbb/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/469573da0fd3b06314dcc83a54493c66b6d169706d8f72be9bc563eb360f9cbb/json HTTP/1.1" 200 586 Stdout:24634 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Postgres Started ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/.env --project-name roottestpostgresqlreplicadatabaseengine1-gw0 --file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_postgres.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/.env --project-name roottestpostgresqlreplicadatabaseengine1-gw0 --file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_postgres.yml up -d --no-recreate] Executing query SYSTEM START MERGES on instance Executing query SYSTEM SYNC REPLICA ON CLUSTER 'cluster' tbl on node1 Stdout:24723 Executing query SYSTEM START FETCHES limited_fetch_table on node2 Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Executing query drop table if exists clone_in_detached sync on instance test_partition/test.py::test_make_clone_in_detached run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Starting new HTTP connection (5): 172.16.9.7:9001 http://172.16.9.7:9001 "GET / HTTP/1.1" 200 0 Connected to Minio. http://172.16.9.7:9001 "GET /root?location= HTTP/1.1" 404 0 [gw1] PASSED test_quota/test.py::test_add_remove_interval Executing query DROP QUOTA IF EXISTS qA, qB on instance test_quota/test.py::test_add_remove_quota http://172.16.9.7:9001 "PUT /root HTTP/1.1" 200 0 S3 bucket 'root' created Stdout:55751 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] http://172.16.9.7:9001 "GET /root2?location= HTTP/1.1" 404 0 Executing query SELECT * FROM tbl ORDER BY id on node1 http://172.16.9.7:9001 "PUT /root2 HTTP/1.1" 200 0 S3 bucket 'root2' created ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env --project-name roottestreplicatedmergetrees3zerocopy-gw8 --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env --project-name roottestreplicatedmergetrees3zerocopy-gw8 --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/docker-compose.yml up -d --no-recreate] Executing query create table clone_in_detached (n int, m String) engine=ReplicatedMergeTree('/clone_in_detached', '1') order by n SETTINGS compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1 on instance Stdout:30204050 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query SELECT * FROM tbl ORDER BY id on node2 Executing query insert into clone_in_detached values (42, '¯-_(ツ)_-¯') on instance Stdout:31833224 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stderr: Container roottestpartlogtable-gw4-node4-1 Stopping Stderr: Container roottestpartlogtable-gw4-node1-1 Stopping Stderr: Container roottestpartlogtable-gw4-node2-1 Stopping Stderr: Container roottestpartlogtable-gw4-node3-1 Stopping Stderr: Container roottestpartlogtable-gw4-node4-1 Stopped Stderr: Container roottestpartlogtable-gw4-node4-1 Removing Stderr: Container roottestpartlogtable-gw4-node3-1 Stopped Stderr: Container roottestpartlogtable-gw4-node3-1 Removing Stderr: Container roottestpartlogtable-gw4-node1-1 Stopped Stderr: Container roottestpartlogtable-gw4-node1-1 Removing Stderr: Container roottestpartlogtable-gw4-node2-1 Stopped Stderr: Container roottestpartlogtable-gw4-node2-1 Removing Stderr: Container roottestpartlogtable-gw4-node2-1 Removed Stderr: Container roottestpartlogtable-gw4-node4-1 Removed Stderr: Container roottestpartlogtable-gw4-node1-1 Removed Stderr: Container roottestpartlogtable-gw4-node3-1 Removed Stderr: Network roottestpartlogtable-gw4_default Removing Stderr: Network roottestpartlogtable-gw4_default Removed Cleanup called thread 2: insert for 2000-01-03: [3, 1, 1, 8, 1, 4, 3, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Docker networks for project roottestpartlogtable-gw4 are NETWORK ID NAME DRIVER SCOPE Executing query SYSTEM RELOAD CONFIG on instance Docker containers for project roottestpartlogtable-gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Stdout:52091 Docker volumes for project roottestpartlogtable-gw4 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestpartlogtable-gw4-.*-1$' --format '{{.ID}}:{{.Names}}'] Executing query CHECK TABLE tbl on node1 Unstopped containers: {} No running containers for project: roottestpartlogtable-gw4 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] thread 4: insert for 2010-01-05: [10, 6, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:4 Command:[docker volume prune -f] thread 0: insert for 2008-01-01: [8, 7, 9, 4, 7, 1, 3, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM RELOAD CONFIG on instance Stdout:Total reclaimed space: 0B Volumes pruned: 4 test_reload_client_certificate/test.py::test_correct_cn_cert Running tests in /ClickHouse/tests/integration/test_reload_client_certificate/test.py Cluster start called. is_up=False thread 1: insert for 2007-01-02: [8, 9, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Docker networks for project roottestreloadclientcertificate-gw4 are NETWORK ID NAME DRIVER SCOPE run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['rm', '/var/lib/clickhouse/data/default/clone_in_detached/all_0_0_0/data.bin'] Command:[docker exec roottestpartition-gw5-instance-1 rm /var/lib/clickhouse/data/default/clone_in_detached/all_0_0_0/data.bin] Docker containers for project roottestreloadclientcertificate-gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Running Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 Creating Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 Created Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 Starting Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 Started ClickHouse instance created get_instance_ip instance_name=instance http://localhost:None "GET /v1.46/containers/roottestpostgresqlreplicadatabaseengine1-gw0-instance-1/json HTTP/1.1" 200 None get_instance_ip instance_name=instance http://localhost:None "GET /v1.46/containers/roottestpostgresqlreplicadatabaseengine1-gw0-instance-1/json HTTP/1.1" 200 None Docker volumes for project roottestreloadclientcertificate-gw4 are DRIVER VOLUME NAME Cleanup called Waiting for ClickHouse start in instance, ip: 172.16.10.3... http://localhost:None "GET /v1.46/containers/roottestpostgresqlreplicadatabaseengine1-gw0-instance-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/9dd2741889aae8e4f1c017690a65ea4583616d64c56da941e0f1a8fe78fe95ea/json HTTP/1.1" 200 None Docker networks for project roottestreloadclientcertificate-gw4 are NETWORK ID NAME DRIVER SCOPE Executing query SELECT * FROM system.quotas ORDER BY name on instance Executing query CHECK TABLE tbl on node2 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['cp', '-r', '/var/lib/clickhouse/data/default/clone_in_detached/all_0_0_0', '/var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_0_0_0'] Command:[docker exec roottestpartition-gw5-instance-1 cp -r /var/lib/clickhouse/data/default/clone_in_detached/all_0_0_0 /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_0_0_0] Docker containers for project roottestreloadclientcertificate-gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreloadclientcertificate-gw4 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreloadclientcertificate-gw4-.*-1$' --format '{{.ID}}:{{.Names}}'] Executing query select * from clone_in_detached on instance Unstopped containers: {} No running containers for project: roottestreloadclientcertificate-gw4 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] http://localhost:None "GET /v1.46/containers/9dd2741889aae8e4f1c017690a65ea4583616d64c56da941e0f1a8fe78fe95ea/json HTTP/1.1" 200 None Stdout:4 Command:[docker volume prune -f] thread 2: insert for 2005-01-03: [9, 6, 9, 6, 6, 9, 10, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:Total reclaimed space: 0B Volumes pruned: 4 Setup directory for instance: node1 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/first_client.crt', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/first_client.key', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/second_client.crt', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/second_client.key', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/third_client.crt', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/third_client.key', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/conf.d/remote_servers.xml', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/conf.d/ssl_conf.xml', '/ClickHouse/tests/integration/test_reload_client_certificate/configs/zookeeper_config_with_ssl.xml'] to /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node1/database Setup logs dir /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node1/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup directory for instance: node2 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/first_client.crt', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/first_client.key', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/second_client.crt', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/second_client.key', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/third_client.crt', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/third_client.key', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/conf.d/remote_servers.xml', '/ClickHouse/tests/integration/test_reload_client_certificate/configs_secure/conf.d/ssl_conf.xml', '/ClickHouse/tests/integration/test_reload_client_certificate/configs/zookeeper_config_with_ssl.xml'] to /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node2/configs/config.d Setup database dir /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node2/database Executing query OPTIMIZE TABLE tbl ON CLUSTER 'cluster' FINAL on node1 Setup logs dir /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node2/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'ZOO_SECURE_CLIENT_PORT': '2281', 'ZK_FS': 'bind', 'ZK_DATA1': '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk1/data', 'ZK_DATA_LOG1': '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk1/log', 'ZK_DATA2': '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk2/data', 'ZK_DATA_LOG2': '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk2/log', 'ZK_DATA3': '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk3/data', 'ZK_DATA_LOG3': '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk3/log'} stored in /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --project-name roottestreloadclientcertificate-gw4 --file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml --file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node2/docker-compose.yml pull] assert_eq_with_retry retry 1 exception Client failed! Return code: 107, stderr: Received exception from server (version 25.2.2): Code: 107. DB::Exception: Received from 172.16.7.2:9000. DB::Exception: Cannot open file /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_0_0_0/data.bin: , errno: 2, strerror: No such file or directory: (while reading from part /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_0_0_0/ in table default.clone_in_detached (f6c1462b-820a-445b-9174-c303fc2a8dd6) located on disk default of type local, from mark 0 with max_rows_to_read = 1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace: 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000cfb8fbb 1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007ff830c 2. DB::ErrnoException::ErrnoException(int, int, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f93d 3. void DB::ErrnoException::throwFromPath(int, String const&, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f4a6 4. DB::OpenedFile::getFD() const @ 0x000000000f823fdc 5. DB::AsynchronousReadBufferFromFileWithDescriptorsCache::AsynchronousReadBufferFromFileWithDescriptorsCache(DB::IAsynchronousReader&, Priority, String const&, unsigned long, int, char*, unsigned long, std::optional, std::shared_ptr) @ 0x000000000f82077c 6. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long)::$_0::operator()(unsigned long, unsigned long, int) const @ 0x000000000f81df3b 7. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long) @ 0x000000000f81db25 8. DB::DiskLocal::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x00000000105a23e7 9. DB::DataPartStorageOnDiskFull::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x0000000011f1d5b0 10. DB::MergeTreeReaderStream::init() @ 0x000000001220806b 11. DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x0000000012206446 12. DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x00000000121f8bc6 13. DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x00000000122011d2 14. DB::MergeTreeReadersChain::read(unsigned long, DB::MarkRanges&) @ 0x000000001221bdbf 15. DB::MergeTreeReadTask::read() @ 0x000000001221ac92 16. DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&) @ 0x0000000012c84e6c 17. DB::MergeTreeSelectProcessor::read() @ 0x0000000012223147 18. DB::MergeTreeSource::tryGenerate() @ 0x0000000012c786dc 19. DB::ISource::work() @ 0x0000000012796e07 20. DB::ExecutionThreadContext::executeTask() @ 0x00000000127b3956 21. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000127a6404 22. DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000127a573e 23. void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000127b7278 24. ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000000d09656f 25. void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000000d09d85a 26. ? @ 0x00007f62aa2bfac3 27. ? @ 0x00007f62aa351850 Job's origin stack trace: 0. void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000000d09e4a9 1. void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000000d097898 2. DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x00000000127b5fb3 3. DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x00000000127b64f9 4. DB::TCPHandler::runImpl() @ 0x0000000012708474 5. DB::TCPHandler::run() @ 0x00000000127264d9 6. Poco::Net::TCPServerConnection::start() @ 0x0000000015dda567 7. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015dda9b9 8. Poco::PooledThread::run() @ 0x0000000015da6d3b 9. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015da521d 10. ? @ 0x00007f62aa2bfac3 11. ? @ 0x00007f62aa351850 . (FILE_DOESNT_EXIST) (query: select * from clone_in_detached) Traceback (most recent call last): File "/ClickHouse/tests/integration/helpers/test_tools.py", line 83, in assert_eq_with_retry instance.query( File "/ClickHouse/tests/integration/helpers/cluster.py", line 3565, in query return self.client.query( File "/ClickHouse/tests/integration/helpers/client.py", line 39, in wrap return func(self, *args, **kwargs) File "/ClickHouse/tests/integration/helpers/client.py", line 79, in query ).get_answer() File "/ClickHouse/tests/integration/helpers/client.py", line 248, in get_answer raise QueryRuntimeException( helpers.client.QueryRuntimeException: Client failed! Return code: 107, stderr: Received exception from server (version 25.2.2): Code: 107. DB::Exception: Received from 172.16.7.2:9000. DB::Exception: Cannot open file /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_0_0_0/data.bin: , errno: 2, strerror: No such file or directory: (while reading from part /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_0_0_0/ in table default.clone_in_detached (f6c1462b-820a-445b-9174-c303fc2a8dd6) located on disk default of type local, from mark 0 with max_rows_to_read = 1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace: 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000cfb8fbb 1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007ff830c 2. DB::ErrnoException::ErrnoException(int, int, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f93d 3. void DB::ErrnoException::throwFromPath(int, String const&, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f4a6 4. DB::OpenedFile::getFD() const @ 0x000000000f823fdc 5. DB::AsynchronousReadBufferFromFileWithDescriptorsCache::AsynchronousReadBufferFromFileWithDescriptorsCache(DB::IAsynchronousReader&, Priority, String const&, unsigned long, int, char*, unsigned long, std::optional, std::shared_ptr) @ 0x000000000f82077c 6. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long)::$_0::operator()(unsigned long, unsigned long, int) const @ 0x000000000f81df3b 7. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long) @ 0x000000000f81db25 8. DB::DiskLocal::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x00000000105a23e7 9. DB::DataPartStorageOnDiskFull::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x0000000011f1d5b0 10. DB::MergeTreeReaderStream::init() @ 0x000000001220806b 11. DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x0000000012206446 12. DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x00000000121f8bc6 13. DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x00000000122011d2 14. DB::MergeTreeReadersChain::read(unsigned long, DB::MarkRanges&) @ 0x000000001221bdbf 15. DB::MergeTreeReadTask::read() @ 0x000000001221ac92 16. DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&) @ 0x0000000012c84e6c 17. DB::MergeTreeSelectProcessor::read() @ 0x0000000012223147 18. DB::MergeTreeSource::tryGenerate() @ 0x0000000012c786dc 19. DB::ISource::work() @ 0x0000000012796e07 20. DB::ExecutionThreadContext::executeTask() @ 0x00000000127b3956 21. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000127a6404 22. DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000127a573e 23. void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000127b7278 24. ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000000d09656f 25. void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000000d09d85a 26. ? @ 0x00007f62aa2bfac3 27. ? @ 0x00007f62aa351850 Job's origin stack trace: 0. void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000000d09e4a9 1. void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000000d097898 2. DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x00000000127b5fb3 3. DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x00000000127b64f9 4. DB::TCPHandler::runImpl() @ 0x0000000012708474 5. DB::TCPHandler::run() @ 0x00000000127264d9 6. Poco::Net::TCPServerConnection::start() @ 0x0000000015dda567 7. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015dda9b9 8. Poco::PooledThread::run() @ 0x0000000015da6d3b 9. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015da521d 10. ? @ 0x00007f62aa2bfac3 11. ? @ 0x00007f62aa351850 . (FILE_DOESNT_EXIST) (query: select * from clone_in_detached) Executing query SELECT quota_name, quota_key, is_current, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows, result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quotas_usage ORDER BY quota_name, quota_key, duration on instance http://localhost:None "GET /v1.46/containers/9dd2741889aae8e4f1c017690a65ea4583616d64c56da941e0f1a8fe78fe95ea/json HTTP/1.1" 200 None thread 0: insert for 2006-01-01: [4, 5, 5, 9, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2000-01-05: [4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Running Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Running Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Running Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Running Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Running Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Running Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node2-1 Creating Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node1-1 Creating Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Running Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node3-1 Creating Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node3-1 Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node2-1 Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node1-1 Created Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node1-1 Starting Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node2-1 Starting Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node3-1 Starting Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node1-1 Started Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node3-1 Started Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node2-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.9.9... http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/9dd2741889aae8e4f1c017690a65ea4583616d64c56da941e0f1a8fe78fe95ea/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/a4ce9da6724ad5f3e1c2af37267e8772bb0265014a296843ec777930bef46cdd/json HTTP/1.1" 200 None run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxkdXJhdGlvbj4zMTU1Njk1MjwvZHVyYXRpb24+CiAgICAgICAgICAgICAgICA8cXVlcmllcz4xMDAwPC9xdWVyaWVzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgoKICAgICAgICA8bXlRdW90YTI+CiAgICAgICAgICAgIDxrZXllZC8+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxyYW5kb21pemU+dHJ1ZTwvcmFuZG9taXplPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjM2MDA8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz40MDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+NDAwMDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVhZF9ieXRlcz40MDAwMDA8L3JlYWRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8cmVzdWx0X2J5dGVzPjQwMDAwMDwvcmVzdWx0X2J5dGVzPgogICAgICAgICAgICAgICAgPGV4ZWN1dGlvbl90aW1lPjYwPC9leGVjdXRpb25fdGltZT4KICAgICAgICAgICAgICAgIDxmYWlsZWRfc2VxdWVudGlhbF9hdXRoZW50aWNhdGlvbnM+MzwvZmFpbGVkX3NlcXVlbnRpYWxfYXV0aGVudGljYXRpb25zPgogICAgICAgICAgICA8L2ludGVydmFsPgogICAgICAgICAgICA8aW50ZXJ2YWwyPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjI2Mjk3NDY8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPGV4ZWN1dGlvbl90aW1lPjE4MDA8L2V4ZWN1dGlvbl90aW1lPgogICAgICAgICAgICA8L2ludGVydmFsMj4KICAgICAgICA8L215UXVvdGEyPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxkdXJhdGlvbj4zMTU1Njk1MjwvZHVyYXRpb24+CiAgICAgICAgICAgICAgICA8cXVlcmllcz4xMDAwPC9xdWVyaWVzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgoKICAgICAgICA8bXlRdW90YTI+CiAgICAgICAgICAgIDxrZXllZC8+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxyYW5kb21pemU+dHJ1ZTwvcmFuZG9taXplPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjM2MDA8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz40MDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+NDAwMDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVhZF9ieXRlcz40MDAwMDA8L3JlYWRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8cmVzdWx0X2J5dGVzPjQwMDAwMDwvcmVzdWx0X2J5dGVzPgogICAgICAgICAgICAgICAgPGV4ZWN1dGlvbl90aW1lPjYwPC9leGVjdXRpb25fdGltZT4KICAgICAgICAgICAgICAgIDxmYWlsZWRfc2VxdWVudGlhbF9hdXRoZW50aWNhdGlvbnM+MzwvZmFpbGVkX3NlcXVlbnRpYWxfYXV0aGVudGljYXRpb25zPgogICAgICAgICAgICA8L2ludGVydmFsPgogICAgICAgICAgICA8aW50ZXJ2YWwyPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjI2Mjk3NDY8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPGV4ZWN1dGlvbl90aW1lPjE4MDA8L2V4ZWN1dGlvbl90aW1lPgogICAgICAgICAgICA8L2ludGVydmFsMj4KICAgICAgICA8L215UXVvdGEyPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Stdout:59981 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 1: insert for 2004-01-02: [3, 6, 8, 4, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM RELOAD CONFIG on instance Executing query SYSTEM SYNC REPLICA ON CLUSTER 'cluster' tbl on node1 http://localhost:None "GET /v1.46/containers/9dd2741889aae8e4f1c017690a65ea4583616d64c56da941e0f1a8fe78fe95ea/json HTTP/1.1" 200 None ClickHouse instance started http://localhost:None "GET /v1.46/containers/a4ce9da6724ad5f3e1c2af37267e8772bb0265014a296843ec777930bef46cdd/json HTTP/1.1" 200 None thread 2: insert for 2003-01-03: [10, 5, 4, 9, 6, 4, 8, 10, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2000-01-04: [3, 1, 8, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:52552270 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query SELECT * FROM system.quotas ORDER BY name on instance Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(1000) on node1 Stdout:52861472 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] http://localhost:None "GET /v1.46/containers/a4ce9da6724ad5f3e1c2af37267e8772bb0265014a296843ec777930bef46cdd/json HTTP/1.1" 200 None Executing query SELECT * FROM tbl ORDER BY id on node1 Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:55793 Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Executing query SELECT * FROM tbl ORDER BY id on node2 Stdout:737 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 4: insert for 2002-01-05: [3, 6, 1, 7, 10, 9, 9, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2001-01-01: [2, 10, 5, 3, 2, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 http://localhost:None "GET /v1.46/containers/a4ce9da6724ad5f3e1c2af37267e8772bb0265014a296843ec777930bef46cdd/json HTTP/1.1" 200 None Executing query DROP DATABASE IF EXISTS `test_database` on instance Stdout:737 Executing query select 20 on node1 Executing query SELECT quota_name, quota_key, is_current, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows, result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quotas_usage ORDER BY quota_name, quota_key, duration on instance Executing query CHECK TABLE tbl on node1 Executing query select * from clone_in_detached on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query CHECK TABLE tbl on node2 http://localhost:None "GET /v1.46/containers/a4ce9da6724ad5f3e1c2af37267e8772bb0265014a296843ec777930bef46cdd/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.9.10... http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/c130b45adbf1994760ccce6e0d11c24ef6fbd0cc8e0bd496d4b3e26475d430f3/json HTTP/1.1" 200 None run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['ls', '/var/lib/clickhouse/data/default/clone_in_detached/detached/'] Command:[docker exec roottestpartition-gw5-instance-1 ls /var/lib/clickhouse/data/default/clone_in_detached/detached/] Executing query SHOW DATABASES on instance Executing query SYSTEM RELOAD CONFIG on instance thread 12: delete 8 * 3 Executing query ALTER TABLE test_mutations DELETE WHERE x = 3 on node2 [gw7] PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids test_replicated_merge_tree_encryption_codec/test.py::test_different_keys run container_id:roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgPC9hZXNfMTI4X2djbV9zaXY+CiAgICA8L2VuY3J5cHRpb25fY29kZWNzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgPC9hZXNfMTI4X2djbV9zaXY+CiAgICA8L2VuY3J5cHRpb25fY29kZWNzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] Stdout:broken_all_0_0_0 Executing query insert into clone_in_detached values (43, '¯-_(ツ)_-¯') on instance thread 1: insert for 2005-01-02: [9, 6, 8, 8, 8, 8, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT * FROM system.quotas ORDER BY name on instance Executing query SYSTEM RELOAD CONFIG on node1 http://localhost:None "GET /v1.46/containers/c130b45adbf1994760ccce6e0d11c24ef6fbd0cc8e0bd496d4b3e26475d430f3/json HTTP/1.1" 200 None ClickHouse node2 started get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-node3-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node3 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-node3-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node3, ip: 172.16.9.11... http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetrees3zerocopy-gw8-node3-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/48383a4aac7e7dddf3eb675be52b1acb924b47154bd6b1bbe2e90cdbae23faaa/json HTTP/1.1" 200 None ClickHouse node3 started Cluster started run container_id:roottestreplicatedmergetrees3zerocopy-gw8-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetrees3zerocopy-gw8-node1-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['rm', '/var/lib/clickhouse/data/default/clone_in_detached/all_1_1_0/data.bin'] Command:[docker exec roottestpartition-gw5-instance-1 rm /var/lib/clickhouse/data/default/clone_in_detached/all_1_1_0/data.bin] thread 3: insert for 2001-01-04: [5, 6, 10, 6, 9, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:1 run container_id:roottestreplicatedmergetrees3zerocopy-gw8-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'kill -HUP 1'] Command:[docker exec -u root roottestreplicatedmergetrees3zerocopy-gw8-node1-1 bash -c kill -HUP 1] Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['cp', '-r', '/var/lib/clickhouse/data/default/clone_in_detached/all_1_1_0', '/var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_1_1_0'] Command:[docker exec roottestpartition-gw5-instance-1 cp -r /var/lib/clickhouse/data/default/clone_in_detached/all_1_1_0 /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_1_1_0] run container_id:roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5iYmJiYmJiYmJiYmJiYmJiPC9rZXk+CiAgICAgICAgPC9hZXNfMTI4X2djbV9zaXY+CiAgICA8L2VuY3J5cHRpb25fY29kZWNzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5iYmJiYmJiYmJiYmJiYmJiPC9rZXk+CiAgICAgICAgPC9hZXNfMTI4X2djbV9zaXY+CiAgICA8L2VuY3J5cHRpb25fY29kZWNzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] thread 2: insert for 2009-01-03: [1, 10, 10, 3, 1, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 4: insert for 2002-01-05: [5, 1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 0: insert for 2003-01-01: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 11: delete 3 * 7 Executing query ALTER TABLE test_mutations DELETE WHERE x = 7 on node1 Executing query SYSTEM RELOAD CONFIG on node2 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['rm', '/var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_1_1_0/primary.idx'] Command:[docker exec roottestpartition-gw5-instance-1 rm /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_1_1_0/primary.idx] run container_id:roottestreplicatedmergetrees3zerocopy-gw8-node2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetrees3zerocopy-gw8-node2-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SELECT quota_name, quota_key, is_current, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows, result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quotas_usage ORDER BY quota_name, quota_key, duration on instance Stdout:1 run container_id:roottestreplicatedmergetrees3zerocopy-gw8-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'kill -HUP 1'] Command:[docker exec -u root roottestreplicatedmergetrees3zerocopy-gw8-node2-1 bash -c kill -HUP 1] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['cp', '-r', '/var/lib/clickhouse/data/default/clone_in_detached/all_1_1_0', '/var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_1_1_0_try0'] Command:[docker exec roottestpartition-gw5-instance-1 cp -r /var/lib/clickhouse/data/default/clone_in_detached/all_1_1_0 /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_1_1_0_try0] Executing query DROP TABLE IF EXISTS tbl ON CLUSTER 'cluster' SYNC on node1 thread 13: delete 12 * 9 Executing query ALTER TABLE test_mutations DELETE WHERE x = 9 on node2 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query select name from system.zookeeper where path='/test/drop_table/replicas' on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "echo 'broken' > /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_1_1_0_try0/checksums.txt"] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Command:[docker exec roottestpartition-gw5-instance-1 bash -c echo 'broken' > /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_1_1_0_try0/checksums.txt] Stdout:62249 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SYSTEM RELOAD CONFIG on instance Executing query select name from system.zookeeper where path='/test/drop_table/replicas' on node2 Executing query select * from clone_in_detached on instance Executing query CREATE TABLE tbl ON CLUSTER 'cluster' ( id Int64, str String Codec(AES_128_GCM_SIV) ) ENGINE=ReplicatedMergeTree('/clickhouse/tables/tbl/', '{replica}') ORDER BY id on node1 Stdout:59367654 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query SELECT * FROM system.quotas ORDER BY name on instance Executing query create table test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3' on node1 assert_eq_with_retry retry 1 exception Client failed! Return code: 107, stderr: Received exception from server (version 25.2.2): Code: 107. DB::Exception: Received from 172.16.7.2:9000. DB::Exception: Cannot open file /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_1_1_0/data.bin: , errno: 2, strerror: No such file or directory: (while reading from part /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_1_1_0/ in table default.clone_in_detached (f6c1462b-820a-445b-9174-c303fc2a8dd6) located on disk default of type local, from mark 0 with max_rows_to_read = 1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace: 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000cfb8fbb 1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007ff830c 2. DB::ErrnoException::ErrnoException(int, int, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f93d 3. void DB::ErrnoException::throwFromPath(int, String const&, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f4a6 4. DB::OpenedFile::getFD() const @ 0x000000000f823fdc 5. DB::AsynchronousReadBufferFromFileWithDescriptorsCache::AsynchronousReadBufferFromFileWithDescriptorsCache(DB::IAsynchronousReader&, Priority, String const&, unsigned long, int, char*, unsigned long, std::optional, std::shared_ptr) @ 0x000000000f82077c 6. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long)::$_0::operator()(unsigned long, unsigned long, int) const @ 0x000000000f81df3b 7. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long) @ 0x000000000f81db25 8. DB::DiskLocal::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x00000000105a23e7 9. DB::DataPartStorageOnDiskFull::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x0000000011f1d5b0 10. DB::MergeTreeReaderStream::init() @ 0x000000001220806b 11. DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x0000000012206446 12. DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x00000000121f8bc6 13. DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x00000000122011d2 14. DB::MergeTreeReadersChain::read(unsigned long, DB::MarkRanges&) @ 0x000000001221bdbf 15. DB::MergeTreeReadTask::read() @ 0x000000001221ac92 16. DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&) @ 0x0000000012c84e6c 17. DB::MergeTreeSelectProcessor::read() @ 0x0000000012223147 18. DB::MergeTreeSource::tryGenerate() @ 0x0000000012c786dc 19. DB::ISource::work() @ 0x0000000012796e07 20. DB::ExecutionThreadContext::executeTask() @ 0x00000000127b3956 21. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000127a6404 22. DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000127a573e 23. void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000127b7278 24. ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000000d09656f 25. void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000000d09d85a 26. ? @ 0x00007f62aa2bfac3 27. ? @ 0x00007f62aa351850 Job's origin stack trace: 0. void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000000d09e4a9 1. void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000000d097898 2. DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x00000000127b5fb3 3. DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x00000000127b64f9 4. DB::TCPHandler::runImpl() @ 0x0000000012708474 5. DB::TCPHandler::run() @ 0x00000000127264d9 6. Poco::Net::TCPServerConnection::start() @ 0x0000000015dda567 7. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015dda9b9 8. Poco::PooledThread::run() @ 0x0000000015da6d3b 9. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015da521d 10. ? @ 0x00007f62aa2bfac3 11. ? @ 0x00007f62aa351850 . (FILE_DOESNT_EXIST) (query: select * from clone_in_detached) Traceback (most recent call last): File "/ClickHouse/tests/integration/helpers/test_tools.py", line 83, in assert_eq_with_retry instance.query( File "/ClickHouse/tests/integration/helpers/cluster.py", line 3565, in query return self.client.query( File "/ClickHouse/tests/integration/helpers/client.py", line 39, in wrap return func(self, *args, **kwargs) File "/ClickHouse/tests/integration/helpers/client.py", line 79, in query ).get_answer() File "/ClickHouse/tests/integration/helpers/client.py", line 248, in get_answer raise QueryRuntimeException( helpers.client.QueryRuntimeException: Client failed! Return code: 107, stderr: Received exception from server (version 25.2.2): Code: 107. DB::Exception: Received from 172.16.7.2:9000. DB::Exception: Cannot open file /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_1_1_0/data.bin: , errno: 2, strerror: No such file or directory: (while reading from part /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_1_1_0/ in table default.clone_in_detached (f6c1462b-820a-445b-9174-c303fc2a8dd6) located on disk default of type local, from mark 0 with max_rows_to_read = 1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace: 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000cfb8fbb 1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007ff830c 2. DB::ErrnoException::ErrnoException(int, int, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f93d 3. void DB::ErrnoException::throwFromPath(int, String const&, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f4a6 4. DB::OpenedFile::getFD() const @ 0x000000000f823fdc 5. DB::AsynchronousReadBufferFromFileWithDescriptorsCache::AsynchronousReadBufferFromFileWithDescriptorsCache(DB::IAsynchronousReader&, Priority, String const&, unsigned long, int, char*, unsigned long, std::optional, std::shared_ptr) @ 0x000000000f82077c 6. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long)::$_0::operator()(unsigned long, unsigned long, int) const @ 0x000000000f81df3b 7. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long) @ 0x000000000f81db25 8. DB::DiskLocal::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x00000000105a23e7 9. DB::DataPartStorageOnDiskFull::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x0000000011f1d5b0 10. DB::MergeTreeReaderStream::init() @ 0x000000001220806b 11. DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x0000000012206446 12. DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x00000000121f8bc6 13. DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x00000000122011d2 14. DB::MergeTreeReadersChain::read(unsigned long, DB::MarkRanges&) @ 0x000000001221bdbf 15. DB::MergeTreeReadTask::read() @ 0x000000001221ac92 16. DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&) @ 0x0000000012c84e6c 17. DB::MergeTreeSelectProcessor::read() @ 0x0000000012223147 18. DB::MergeTreeSource::tryGenerate() @ 0x0000000012c786dc 19. DB::ISource::work() @ 0x0000000012796e07 20. DB::ExecutionThreadContext::executeTask() @ 0x00000000127b3956 21. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000127a6404 22. DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000127a573e 23. void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000127b7278 24. ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000000d09656f 25. void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000000d09d85a 26. ? @ 0x00007f62aa2bfac3 27. ? @ 0x00007f62aa351850 Job's origin stack trace: 0. void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000000d09e4a9 1. void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000000d097898 2. DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x00000000127b5fb3 3. DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x00000000127b64f9 4. DB::TCPHandler::runImpl() @ 0x0000000012708474 5. DB::TCPHandler::run() @ 0x00000000127264d9 6. Poco::Net::TCPServerConnection::start() @ 0x0000000015dda567 7. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015dda9b9 8. Poco::PooledThread::run() @ 0x0000000015da6d3b 9. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015da521d 10. ? @ 0x00007f62aa2bfac3 11. ? @ 0x00007f62aa351850 . (FILE_DOESNT_EXIST) (query: select * from clone_in_detached) Stdout:59479924 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 Stdout:57971 Executing query create table test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '2') order by n partition by n % 99 settings storage_policy='s3' on node2 Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance thread 0: insert for 2010-01-01: [3, 2, 3, 9, 7, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query INSERT INTO tbl VALUES (1, 'str1') on node1 thread 1: insert for 2002-01-02: [5, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query insert into test_drop_table select * from numbers(1000) on node1 Executing query SELECT quota_name, quota_key, is_current, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows, result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quotas_usage ORDER BY quota_name, quota_key, duration on instance thread 4: insert for 2002-01-05: [3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query INSERT INTO tbl VALUES (2, 'str2') on node2 thread 2: insert for 2000-01-03: [4, 9, 7, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2010-01-04: [5, 9, 3, 4, 9, 1, 6, 7, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query SYSTEM SYNC REPLICA ON CLUSTER 'cluster' tbl on node1 Executing query SYSTEM RELOAD CONFIG on instance Executing query SELECT * FROM system.quotas ORDER BY name on instance Executing query SELECT * FROM tbl on node1 Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Executing query SELECT * FROM tbl on node2 thread 4: insert for 2003-01-05: [4, 3, 3, 4, 8, 3, 8, 10, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SELECT quota_name, quota_key, is_current, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows, result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quotas_usage ORDER BY quota_name, quota_key, duration on instance thread 2: insert for 2005-01-03: [4, 8] [gw7] PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_keys test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces run container_id:roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgICAgIDxub25jZT54eHh4eHh4eHh4eHg8L25vbmNlPgogICAgICAgIDwvYWVzXzEyOF9nY21fc2l2PgogICAgPC9lbmNyeXB0aW9uX2NvZGVjcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgICAgIDxub25jZT54eHh4eHh4eHh4eHg8L25vbmNlPgogICAgICAgIDwvYWVzXzEyOF9nY21fc2l2PgogICAgPC9lbmNyeXB0aW9uX2NvZGVjcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 1: insert for 2009-01-02: [4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2006-01-01: [3, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 Executing query select * from clone_in_detached on instance Executing query SYSTEM RELOAD CONFIG on node1 Executing query DROP QUOTA IF EXISTS qA, qB on instance [gw1] PASSED test_quota/test.py::test_add_remove_quota test_quota/test.py::test_consumption_of_show_clusters run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['ls', '/var/lib/clickhouse/data/default/clone_in_detached/detached/'] Command:[docker exec roottestpartition-gw5-instance-1 ls /var/lib/clickhouse/data/default/clone_in_detached/detached/] run container_id:roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgICAgIDxub25jZT55eXl5eXl5eXl5eXk8L25vbmNlPgogICAgICAgIDwvYWVzXzEyOF9nY21fc2l2PgogICAgPC9lbmNyeXB0aW9uX2NvZGVjcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgICAgIDxub25jZT55eXl5eXl5eXl5eXk8L25vbmNlPgogICAgICAgIDwvYWVzXzEyOF9nY21fc2l2PgogICAgPC9lbmNyeXB0aW9uX2NvZGVjcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 3: insert for 2009-01-04: [2, 6, 9, 6, 7, 4, 4, 7, 1, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:64625 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:broken_all_0_0_0 Stdout:broken_all_1_1_0 Stdout:broken_all_1_1_0_try0 Stdout:broken_all_1_1_0_try1 Executing query insert into clone_in_detached values (44, '¯-_(ツ)_-¯') on instance Executing query SYSTEM RELOAD CONFIG on node2 Executing query SYSTEM RELOAD CONFIG on instance Stdout:67449234 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['rm', '/var/lib/clickhouse/data/default/clone_in_detached/all_2_2_0/data.bin'] Command:[docker exec roottestpartition-gw5-instance-1 rm /var/lib/clickhouse/data/default/clone_in_detached/all_2_2_0/data.bin] Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 Stdout:67777708 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query DROP TABLE IF EXISTS tbl ON CLUSTER 'cluster' SYNC on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['cp', '-r', '/var/lib/clickhouse/data/default/clone_in_detached/all_2_2_0', '/var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_2_2_0'] Command:[docker exec roottestpartition-gw5-instance-1 cp -r /var/lib/clickhouse/data/default/clone_in_detached/all_2_2_0 /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_2_2_0] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Stdout:60413 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['rm', '/var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_2_2_0/primary.idx'] Command:[docker exec roottestpartition-gw5-instance-1 rm /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_2_2_0/primary.idx] thread 4: insert for 2008-01-05: [7, 7, 8, 2, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2007-01-03: [5, 8, 5] Executing query SYSTEM RELOAD CONFIG on instance Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2010-01-04: [7] Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 1: insert for 2009-01-02: [2, 5, 5, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['cp', '-r', '/var/lib/clickhouse/data/default/clone_in_detached/all_2_2_0', '/var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_2_2_0_try0'] Command:[docker exec roottestpartition-gw5-instance-1 cp -r /var/lib/clickhouse/data/default/clone_in_detached/all_2_2_0 /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_2_2_0_try0] Executing query CREATE TABLE tbl ON CLUSTER 'cluster' ( id Int64, str String Codec(AES_128_GCM_SIV) ) ENGINE=ReplicatedMergeTree('/clickhouse/tables/tbl/', '{replica}') ORDER BY id on node1 thread 0: insert for 2001-01-01: [5, 3, 7, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "echo 'broken' > /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_2_2_0_try0/checksums.txt"] Command:[docker exec roottestpartition-gw5-instance-1 bash -c echo 'broken' > /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_2_2_0_try0/checksums.txt] Executing query SHOW CLUSTERS on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['cp', '-r', '/var/lib/clickhouse/data/default/clone_in_detached/all_2_2_0', '/var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_2_2_0_try1'] Command:[docker exec roottestpartition-gw5-instance-1 cp -r /var/lib/clickhouse/data/default/clone_in_detached/all_2_2_0 /var/lib/clickhouse/data/default/clone_in_detached/detached/broken_all_2_2_0_try1] Executing query SHOW QUOTA on instance Executing query select * from clone_in_detached on instance Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 Executing query DROP QUOTA IF EXISTS qA, qB on instance [gw1] PASSED test_quota/test.py::test_consumption_of_show_clusters test_quota/test.py::test_consumption_of_show_databases Executing query INSERT INTO tbl VALUES (1, 'str1') on node1 assert_eq_with_retry retry 1 exception Client failed! Return code: 107, stderr: Received exception from server (version 25.2.2): Code: 107. DB::Exception: Received from 172.16.7.2:9000. DB::Exception: Cannot open file /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_2_2_0/data.bin: , errno: 2, strerror: No such file or directory: (while reading from part /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_2_2_0/ in table default.clone_in_detached (f6c1462b-820a-445b-9174-c303fc2a8dd6) located on disk default of type local, from mark 0 with max_rows_to_read = 1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace: 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000cfb8fbb 1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007ff830c 2. DB::ErrnoException::ErrnoException(int, int, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f93d 3. void DB::ErrnoException::throwFromPath(int, String const&, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f4a6 4. DB::OpenedFile::getFD() const @ 0x000000000f823fdc 5. DB::AsynchronousReadBufferFromFileWithDescriptorsCache::AsynchronousReadBufferFromFileWithDescriptorsCache(DB::IAsynchronousReader&, Priority, String const&, unsigned long, int, char*, unsigned long, std::optional, std::shared_ptr) @ 0x000000000f82077c 6. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long)::$_0::operator()(unsigned long, unsigned long, int) const @ 0x000000000f81df3b 7. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long) @ 0x000000000f81db25 8. DB::DiskLocal::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x00000000105a23e7 9. DB::DataPartStorageOnDiskFull::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x0000000011f1d5b0 10. DB::MergeTreeReaderStream::init() @ 0x000000001220806b 11. DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x0000000012206446 12. DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x00000000121f8bc6 13. DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x00000000122011d2 14. DB::MergeTreeReadersChain::read(unsigned long, DB::MarkRanges&) @ 0x000000001221bdbf 15. DB::MergeTreeReadTask::read() @ 0x000000001221ac92 16. DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&) @ 0x0000000012c84e6c 17. DB::MergeTreeSelectProcessor::read() @ 0x0000000012223147 18. DB::MergeTreeSource::tryGenerate() @ 0x0000000012c786dc 19. DB::ISource::work() @ 0x0000000012796e07 20. DB::ExecutionThreadContext::executeTask() @ 0x00000000127b3956 21. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000127a6404 22. DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000127a573e 23. void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000127b7278 24. ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000000d09656f 25. void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000000d09d85a 26. ? @ 0x00007f62aa2bfac3 27. ? @ 0x00007f62aa351850 Job's origin stack trace: 0. void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000000d09e4a9 1. void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000000d097898 2. DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x00000000127b5fb3 3. DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x00000000127b64f9 4. DB::TCPHandler::runImpl() @ 0x0000000012708474 5. DB::TCPHandler::run() @ 0x00000000127264d9 6. Poco::Net::TCPServerConnection::start() @ 0x0000000015dda567 7. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015dda9b9 8. Poco::PooledThread::run() @ 0x0000000015da6d3b 9. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015da521d 10. ? @ 0x00007f62aa2bfac3 11. ? @ 0x00007f62aa351850 . (FILE_DOESNT_EXIST) (query: select * from clone_in_detached) Traceback (most recent call last): File "/ClickHouse/tests/integration/helpers/test_tools.py", line 83, in assert_eq_with_retry instance.query( File "/ClickHouse/tests/integration/helpers/cluster.py", line 3565, in query return self.client.query( File "/ClickHouse/tests/integration/helpers/client.py", line 39, in wrap return func(self, *args, **kwargs) File "/ClickHouse/tests/integration/helpers/client.py", line 79, in query ).get_answer() File "/ClickHouse/tests/integration/helpers/client.py", line 248, in get_answer raise QueryRuntimeException( helpers.client.QueryRuntimeException: Client failed! Return code: 107, stderr: Received exception from server (version 25.2.2): Code: 107. DB::Exception: Received from 172.16.7.2:9000. DB::Exception: Cannot open file /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_2_2_0/data.bin: , errno: 2, strerror: No such file or directory: (while reading from part /var/lib/clickhouse/store/f6c/f6c1462b-820a-445b-9174-c303fc2a8dd6/all_2_2_0/ in table default.clone_in_detached (f6c1462b-820a-445b-9174-c303fc2a8dd6) located on disk default of type local, from mark 0 with max_rows_to_read = 1): While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). Stack trace: 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000cfb8fbb 1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007ff830c 2. DB::ErrnoException::ErrnoException(int, int, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f93d 3. void DB::ErrnoException::throwFromPath(int, String const&, FormatStringHelperImpl::type>, String const&) @ 0x000000000d00f4a6 4. DB::OpenedFile::getFD() const @ 0x000000000f823fdc 5. DB::AsynchronousReadBufferFromFileWithDescriptorsCache::AsynchronousReadBufferFromFileWithDescriptorsCache(DB::IAsynchronousReader&, Priority, String const&, unsigned long, int, char*, unsigned long, std::optional, std::shared_ptr) @ 0x000000000f82077c 6. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long)::$_0::operator()(unsigned long, unsigned long, int) const @ 0x000000000f81df3b 7. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional, std::optional, int, char*, unsigned long) @ 0x000000000f81db25 8. DB::DiskLocal::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x00000000105a23e7 9. DB::DataPartStorageOnDiskFull::readFile(String const&, DB::ReadSettings const&, std::optional, std::optional) const @ 0x0000000011f1d5b0 10. DB::MergeTreeReaderStream::init() @ 0x000000001220806b 11. DB::MergeTreeReaderCompactSingleBuffer::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x0000000012206446 12. DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector::immutable_ptr, std::allocator::immutable_ptr>>&) @ 0x00000000121f8bc6 13. DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x00000000122011d2 14. DB::MergeTreeReadersChain::read(unsigned long, DB::MarkRanges&) @ 0x000000001221bdbf 15. DB::MergeTreeReadTask::read() @ 0x000000001221ac92 16. DB::MergeTreeInOrderSelectAlgorithm::readFromTask(DB::MergeTreeReadTask&) @ 0x0000000012c84e6c 17. DB::MergeTreeSelectProcessor::read() @ 0x0000000012223147 18. DB::MergeTreeSource::tryGenerate() @ 0x0000000012c786dc 19. DB::ISource::work() @ 0x0000000012796e07 20. DB::ExecutionThreadContext::executeTask() @ 0x00000000127b3956 21. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic*) @ 0x00000000127a6404 22. DB::PipelineExecutor::execute(unsigned long, bool) @ 0x00000000127a573e 23. void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl(DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x00000000127b7278 24. ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000000d09656f 25. void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000000d09d85a 26. ? @ 0x00007f62aa2bfac3 27. ? @ 0x00007f62aa351850 Job's origin stack trace: 0. void boost::heap::priority_queue, boost::parameter::void_, boost::parameter::void_, boost::parameter::void_>::emplace, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const, bool&, (anonymous namespace)::ScopedDecrement>(std::function&&, Priority&, StrongTypedef&, DB::OpenTelemetry::TracingContextOnThread const&&, bool&, (anonymous namespace)::ScopedDecrement&&) @ 0x000000000d09e4a9 1. void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool) @ 0x000000000d097898 2. DB::PullingAsyncPipelineExecutor::pull(DB::Chunk&, unsigned long) @ 0x00000000127b5fb3 3. DB::PullingAsyncPipelineExecutor::pull(DB::Block&, unsigned long) @ 0x00000000127b64f9 4. DB::TCPHandler::runImpl() @ 0x0000000012708474 5. DB::TCPHandler::run() @ 0x00000000127264d9 6. Poco::Net::TCPServerConnection::start() @ 0x0000000015dda567 7. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015dda9b9 8. Poco::PooledThread::run() @ 0x0000000015da6d3b 9. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015da521d 10. ? @ 0x00007f62aa2bfac3 11. ? @ 0x00007f62aa351850 . (FILE_DOESNT_EXIST) (query: select * from clone_in_detached) thread 3: insert for 2003-01-04: [8, 7, 10, 8, 3, 4, 9] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO tbl VALUES (1, 'str1') on node2 thread 1: insert for 2007-01-02: [6, 2, 3, 5, 5, 4, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM RELOAD CONFIG on instance Executing query INSERT INTO tbl VALUES (2, 'str2') on node2 thread 4: insert for 2002-01-05: [4] thread 2: insert for 2004-01-03: [9, 8, 8, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 0: insert for 2008-01-01: [10, 5, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 Executing query SYSTEM SYNC REPLICA ON CLUSTER 'cluster' tbl on node1 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Stdout:66869 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SYSTEM RELOAD CONFIG on instance Stdout:75401114 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query SELECT * FROM tbl ORDER BY id on node1 Executing query CREATE TABLE test_add_disk ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 thread 3: insert for 2002-01-04: [10, 2, 6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:75781078 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SHOW DATABASES on instance Executing query SELECT count() FROM test_rename_distributed WHERE foo2 % 1000 > 0 on node1 Executing query SELECT * FROM tbl ORDER BY id on node2 Executing query SELECT name FROM system.disks on node1 Stdout:62801 Executing query SHOW QUOTA on instance Executing query CHECK TABLE tbl on node1 Executing query SYSTEM RELOAD CONFIG on node1 thread 2: insert for 2006-01-03: [5, 7, 3, 5, 4, 5, 3, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query select * from clone_in_detached on instance [gw1] PASSED test_quota/test.py::test_consumption_of_show_databases test_quota/test.py::test_consumption_of_show_privileges Executing query DROP QUOTA IF EXISTS qA, qB on instance Executing query CHECK TABLE tbl on node2 Executing query SELECT name FROM system.disks on node1 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 1: insert for 2000-01-02: [2, 8, 9, 2, 4, 4, 9, 1, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query DROP TABLE IF EXISTS test_rename_distributed ON CLUSTER test_cluster SYNC on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['ls', '/var/lib/clickhouse/data/default/clone_in_detached/detached/'] Command:[docker exec roottestpartition-gw5-instance-1 ls /var/lib/clickhouse/data/default/clone_in_detached/detached/] Executing query SYSTEM RELOAD CONFIG on instance Executing query OPTIMIZE TABLE tbl ON CLUSTER 'cluster' FINAL on node1 thread 0: insert for 2001-01-01: [9, 4, 10, 5, 5, 8, 4, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query DROP TABLE IF EXISTS test_add_disk on node1 Stdout:broken_all_0_0_0 Stdout:broken_all_1_1_0 Stdout:broken_all_1_1_0_try0 Stdout:broken_all_1_1_0_try1 Stdout:broken_all_2_2_0 Stdout:broken_all_2_2_0_try0 Stdout:broken_all_2_2_0_try1 [gw5] PASSED test_partition/test.py::test_make_clone_in_detached Executing query DROP TABLE IF EXISTS test.partition_complex on instance test_partition/test.py::test_partition_complex run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 4: insert for 2008-01-05: [5, 1, 7, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 [gw2] PASSED test_reloading_storage_configuration/test.py::test_add_disk Executing query DROP TABLE IF EXISTS test_rename_distributed_replicated ON CLUSTER test_cluster SYNC on node1 test_reloading_storage_configuration/test.py::test_add_disk_to_policy run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] Executing query CREATE TABLE test.partition_complex (p Date, k Int8, v1 Int8 MATERIALIZED k + 1) ENGINE = MergeTree PARTITION BY p ORDER BY k SETTINGS index_granularity=1, index_granularity_bytes=0, compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1, replace_long_file_name_to_hash=false on instance Executing query SYSTEM RELOAD CONFIG on instance thread 3: insert for 2000-01-04: [10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM SYNC REPLICA ON CLUSTER 'cluster' tbl on node1 thread 11: delete 28 * 4 Executing query ALTER TABLE test_mutations DELETE WHERE x = 4 on node1 Stdout: PID TTY TIME CMD Stdout: 737 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] Executing query INSERT INTO test.partition_complex (p, k) VALUES(toDate(31), 1) on instance Executing query SHOW PRIVILEGES on instance Executing query INSERT INTO test.partition_complex (p, k) VALUES(toDate(1), 2) on instance run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SHOW QUOTA on instance run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query SELECT * FROM tbl ORDER BY id on node1 thread 1: insert for 2005-01-02: [6, 1, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 2: insert for 2007-01-03: [6, 7, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SELECT name FROM system.parts WHERE database='test' AND table='partition_complex' on instance Stdout:737 Stdout:69199 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 4: insert for 2001-01-05: [7, 5, 6, 2, 2, 1, 2, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query DROP QUOTA IF EXISTS qA, qB on instance [gw1] PASSED test_quota/test.py::test_consumption_of_show_privileges test_quota/test.py::test_consumption_of_show_processlist Stdout:83090040 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query SELECT * FROM tbl ORDER BY id on node2 thread 0: insert for 2002-01-01: [10, 2, 3, 4, 4, 4, 8, 10, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['wc', '-l', '/var/lib/clickhouse/data/test/partition_complex/19700102_2_2_0/columns.txt'] Command:[docker exec roottestpartition-gw5-instance-1 wc -l /var/lib/clickhouse/data/test/partition_complex/19700102_2_2_0/columns.txt] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query CHECK TABLE tbl on node1 Stdout:83705148 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SYSTEM RELOAD CONFIG on instance Stdout:5 /var/lib/clickhouse/data/test/partition_complex/19700102_2_2_0/columns.txt run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['wc', '-l', '/var/lib/clickhouse/data/test/partition_complex/19700201_1_1_0/columns.txt'] Command:[docker exec roottestpartition-gw5-instance-1 wc -l /var/lib/clickhouse/data/test/partition_complex/19700201_1_1_0/columns.txt] Executing query CHECK TABLE tbl on node2 Stdout:65111 thread 3: insert for 2003-01-04: [1, 6, 9, 9, 6, 9, 10, 6, 3, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:5 /var/lib/clickhouse/data/test/partition_complex/19700201_1_1_0/columns.txt Executing query ALTER TABLE test.partition_complex FREEZE on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] [gw7] PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces test_replicated_merge_tree_encryption_codec/test.py::test_same_keys run container_id:roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgPC9hZXNfMTI4X2djbV9zaXY+CiAgICA8L2VuY3J5cHRpb25fY29kZWNzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Executing query system sync replica test_drop_table on node2 Command:[docker exec roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgPC9hZXNfMTI4X2djbV9zaXY+CiAgICA8L2VuY3J5cHRpb25fY29kZWNzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] thread 13: delete 27 * 8 Executing query ALTER TABLE test_mutations DELETE WHERE x = 8 on node1 Executing query SYSTEM RELOAD CONFIG on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "cd /var/lib/clickhouse/ && find shadow -type f -exec md5sum {} \\; | grep partition_complex | sed 's shadow/[0-9]*/data/[a-z0-9_-]*/ shadow/1/data/test/ g' | sort | uniq"] Command:[docker exec roottestpartition-gw5-instance-1 bash -c cd /var/lib/clickhouse/ && find shadow -type f -exec md5sum {} \; | grep partition_complex | sed 's shadow/[0-9]*/data/[a-z0-9_-]*/ shadow/1/data/test/ g' | sort | uniq] Executing query SYSTEM RELOAD CONFIG on node1 Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found thread 1: insert for 2007-01-02: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 http://localhost:None "GET /version HTTP/1.1" 200 826 Executing query SHOW PROCESSLIST on instance thread 0: insert for 2010-01-01: [1, 4, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:082814b5aa5109160d5c0c5aff10d4df shadow/1/data/test/partition_complex/19700102_2_2_0/k.bin Stdout:082814b5aa5109160d5c0c5aff10d4df shadow/1/data/test/partition_complex/19700201_1_1_0/v1.bin Stdout:13cae8e658e0ca4f75c56b1fc424e150 shadow/1/data/test/partition_complex/19700102_2_2_0/minmax_p.idx Stdout:25daad3d9e60b45043a70c4ab7d3b1c6 shadow/1/data/test/partition_complex/19700102_2_2_0/partition.dat Stdout:3726312af62aec86b64a7708d5751787 shadow/1/data/test/partition_complex/19700201_1_1_0/partition.dat Stdout:37855b06a39b79a67ea4e86e4a3299aa shadow/1/data/test/partition_complex/19700102_2_2_0/checksums.txt Stdout:38e62ff37e1e5064e9a3f605dfe09d13 shadow/1/data/test/partition_complex/19700102_2_2_0/v1.bin Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700102_2_2_0/k.mrk Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700102_2_2_0/p.mrk Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700102_2_2_0/v1.mrk Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700201_1_1_0/k.mrk Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700201_1_1_0/p.mrk Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700201_1_1_0/v1.mrk Stdout:55a54008ad1ba589aa210d2629c1df41 shadow/1/data/test/partition_complex/19700201_1_1_0/primary.idx Stdout:5f087cb3e7071bf9407e095821e2af8f shadow/1/data/test/partition_complex/19700201_1_1_0/checksums.txt Stdout:77d5af402ada101574f4da114f242e02 shadow/1/data/test/partition_complex/19700102_2_2_0/columns.txt Stdout:77d5af402ada101574f4da114f242e02 shadow/1/data/test/partition_complex/19700201_1_1_0/columns.txt Stdout:88cdc31ded355e7572d68d8cde525d3a shadow/1/data/test/partition_complex/19700201_1_1_0/p.bin Stdout:9e688c58a5487b8eaf69c9e1005ad0bf shadow/1/data/test/partition_complex/19700102_2_2_0/primary.idx Stdout:c0904274faa8f3f06f35666cc9c5bd2f shadow/1/data/test/partition_complex/19700102_2_2_0/default_compression_codec.txt Stdout:c0904274faa8f3f06f35666cc9c5bd2f shadow/1/data/test/partition_complex/19700201_1_1_0/default_compression_codec.txt Stdout:c4ca4238a0b923820dcc509a6f75849b shadow/1/data/test/partition_complex/19700102_2_2_0/count.txt Stdout:c4ca4238a0b923820dcc509a6f75849b shadow/1/data/test/partition_complex/19700201_1_1_0/count.txt Stdout:cfcb770c3ecd0990dcceb1bde129e6c6 shadow/1/data/test/partition_complex/19700102_2_2_0/p.bin Stdout:e2af3bef1fd129aea73a890ede1e7a30 shadow/1/data/test/partition_complex/19700201_1_1_0/k.bin Stdout:f2312862cc01adf34a93151377be2ddf shadow/1/data/test/partition_complex/19700201_1_1_0/minmax_p.idx Executing query ALTER TABLE test.partition_complex DETACH PARTITION 197001 on instance thread 2: insert for 2009-01-03: [7, 9, 2, 4, 3, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgPC9hZXNfMTI4X2djbV9zaXY+CiAgICA8L2VuY3J5cHRpb25fY29kZWNzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8ZW5jcnlwdGlvbl9jb2RlY3M+CiAgICAgICAgPGFlc18xMjhfZ2NtX3Npdj4KICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgPC9hZXNfMTI4X2djbV9zaXY+CiAgICA8L2VuY3J5cHRpb25fY29kZWNzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] Executing query ALTER TABLE test.partition_complex ATTACH PARTITION 197001 on instance Executing query SYSTEM RELOAD CONFIG on node2 thread 4: insert for 2002-01-05: [7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SHOW QUOTA on instance thread 12: delete 24 * 6 Executing query ALTER TABLE test_mutations DELETE WHERE x = 6 on node2 Executing query SELECT name FROM system.parts WHERE database='test' AND table='partition_complex' on instance Executing query DROP TABLE IF EXISTS tbl ON CLUSTER 'cluster' SYNC on node1 http://localhost:None "POST /v1.46/containers/create HTTP/1.1" 201 88 http://localhost:None "GET /v1.46/containers/322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62/json HTTP/1.1" 200 None run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['wc', '-l', '/var/lib/clickhouse/data/test/partition_complex/19700102_2_2_0/columns.txt'] Command:[docker exec roottestpartition-gw5-instance-1 wc -l /var/lib/clickhouse/data/test/partition_complex/19700102_2_2_0/columns.txt] [gw1] PASSED test_quota/test.py::test_consumption_of_show_processlist Executing query DROP QUOTA IF EXISTS qA, qB on instance test_quota/test.py::test_consumption_of_show_tables thread 3: insert for 2002-01-04: [5, 7, 4, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:5 /var/lib/clickhouse/data/test/partition_complex/19700102_2_2_0/columns.txt run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['wc', '-l', '/var/lib/clickhouse/data/test/partition_complex/19700201_1_1_0/columns.txt'] Command:[docker exec roottestpartition-gw5-instance-1 wc -l /var/lib/clickhouse/data/test/partition_complex/19700201_1_1_0/columns.txt] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Stdout:5 /var/lib/clickhouse/data/test/partition_complex/19700201_1_1_0/columns.txt Executing query ALTER TABLE test.partition_complex MODIFY COLUMN v1 Int8 on instance Executing query SYSTEM RELOAD CONFIG on instance http://localhost:None "POST /v1.46/containers/322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62/start HTTP/1.1" 204 0 [network] Created new container 322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62 http://localhost:None "POST /v1.46/containers/322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/f7f495ded5ed5938ea5d73a479c72adacd8fd1be56fc87d1484dbc98a75d7583/start HTTP/1.1" 101 0 thread 2: insert for 2006-01-03: [7, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] http://localhost:None "GET /v1.46/exec/f7f495ded5ed5938ea5d73a479c72adacd8fd1be56fc87d1484dbc98a75d7583/json HTTP/1.1" 200 548 [network] 322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62: ['iptables', '--wait', '-I', 'DOCKER-USER', '1', '-m', 'statistic', '--mode', 'random', '--probability', '0.01', '-p', 'tcp', '-d', '172.16.9.9', '--sport', '2181', '-j', 'REJECT', '--reject-with', 'tcp-reset'] (0): thread 1: insert for 2007-01-02: [8, 4, 10, 5, 6, 9, 2, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 http://localhost:None "POST /v1.46/containers/322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/d1bd456af840dd87f65c7b982ed578eebc3a4ceb86933c9ac3eadbaaa09ecb03/start HTTP/1.1" 101 0 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "cd /var/lib/clickhouse/ && find shadow -type f -exec md5sum {} \\; | grep partition_complex | sed 's shadow/[0-9]*/data/[a-z0-9_-]*/ shadow/1/data/test/ g' | sort | uniq"] Command:[docker exec roottestpartition-gw5-instance-1 bash -c cd /var/lib/clickhouse/ && find shadow -type f -exec md5sum {} \; | grep partition_complex | sed 's shadow/[0-9]*/data/[a-z0-9_-]*/ shadow/1/data/test/ g' | sort | uniq] Executing query CREATE TABLE tbl ON CLUSTER 'cluster' ( id Int64, str String Codec(AES_128_GCM_SIV) ) ENGINE=ReplicatedMergeTree('/clickhouse/tables/tbl/', '{replica}') ORDER BY id on node1 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 0: insert for 2002-01-01: [3, 10, 7, 5, 5, 9, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:71509 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] http://localhost:None "GET /v1.46/exec/d1bd456af840dd87f65c7b982ed578eebc3a4ceb86933c9ac3eadbaaa09ecb03/json HTTP/1.1" 200 548 [network] 322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62: ['iptables', '--wait', '-I', 'DOCKER-USER', '1', '-m', 'statistic', '--mode', 'random', '--probability', '0.01', '-p', 'tcp', '-s', '172.16.9.9', '--dport', '2181', '-j', 'REJECT', '--reject-with', 'tcp-reset'] (0): Executing query drop table test_drop_table on node1 Executing query SYSTEM RELOAD CONFIG on instance thread 4: insert for 2001-01-05: [9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:082814b5aa5109160d5c0c5aff10d4df shadow/1/data/test/partition_complex/19700102_2_2_0/k.bin Stdout:082814b5aa5109160d5c0c5aff10d4df shadow/1/data/test/partition_complex/19700201_1_1_0/v1.bin Stdout:13cae8e658e0ca4f75c56b1fc424e150 shadow/1/data/test/partition_complex/19700102_2_2_0/minmax_p.idx Stdout:25daad3d9e60b45043a70c4ab7d3b1c6 shadow/1/data/test/partition_complex/19700102_2_2_0/partition.dat Stdout:3726312af62aec86b64a7708d5751787 shadow/1/data/test/partition_complex/19700201_1_1_0/partition.dat Stdout:37855b06a39b79a67ea4e86e4a3299aa shadow/1/data/test/partition_complex/19700102_2_2_0/checksums.txt Stdout:38e62ff37e1e5064e9a3f605dfe09d13 shadow/1/data/test/partition_complex/19700102_2_2_0/v1.bin Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700102_2_2_0/k.mrk Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700102_2_2_0/p.mrk Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700102_2_2_0/v1.mrk Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700201_1_1_0/k.mrk Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700201_1_1_0/p.mrk Stdout:4ae71336e44bf9bf79d2752e234818a5 shadow/1/data/test/partition_complex/19700201_1_1_0/v1.mrk Stdout:55a54008ad1ba589aa210d2629c1df41 shadow/1/data/test/partition_complex/19700201_1_1_0/primary.idx Stdout:5f087cb3e7071bf9407e095821e2af8f shadow/1/data/test/partition_complex/19700201_1_1_0/checksums.txt Stdout:77d5af402ada101574f4da114f242e02 shadow/1/data/test/partition_complex/19700102_2_2_0/columns.txt Stdout:77d5af402ada101574f4da114f242e02 shadow/1/data/test/partition_complex/19700201_1_1_0/columns.txt Stdout:88cdc31ded355e7572d68d8cde525d3a shadow/1/data/test/partition_complex/19700201_1_1_0/p.bin Stdout:9e688c58a5487b8eaf69c9e1005ad0bf shadow/1/data/test/partition_complex/19700102_2_2_0/primary.idx Stdout:c0904274faa8f3f06f35666cc9c5bd2f shadow/1/data/test/partition_complex/19700102_2_2_0/default_compression_codec.txt Stdout:c0904274faa8f3f06f35666cc9c5bd2f shadow/1/data/test/partition_complex/19700201_1_1_0/default_compression_codec.txt Stdout:c4ca4238a0b923820dcc509a6f75849b shadow/1/data/test/partition_complex/19700102_2_2_0/count.txt Stdout:c4ca4238a0b923820dcc509a6f75849b shadow/1/data/test/partition_complex/19700201_1_1_0/count.txt Stdout:cfcb770c3ecd0990dcceb1bde129e6c6 shadow/1/data/test/partition_complex/19700102_2_2_0/p.bin Stdout:e2af3bef1fd129aea73a890ede1e7a30 shadow/1/data/test/partition_complex/19700201_1_1_0/k.bin Stdout:f2312862cc01adf34a93151377be2ddf shadow/1/data/test/partition_complex/19700201_1_1_0/minmax_p.idx Executing query OPTIMIZE TABLE test.partition_complex on instance Stdout:91142418 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query SHOW TABLES on instance Stdout:91522382 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SELECT toUInt16(p), k, v1 FROM test.partition_complex ORDER BY k on instance Executing query INSERT INTO tbl VALUES (1, 'str1') on node1 Stdout:67289 Executing query SHOW QUOTA on instance thread 11: delete 23 * 1 Executing query ALTER TABLE test_mutations DELETE WHERE x = 1 on node2 Executing query INSERT INTO tbl VALUES (1, 'str1') on node2 thread 3: insert for 2008-01-04: [7, 3, 9, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] [gw1] PASSED test_quota/test.py::test_consumption_of_show_tables Executing query DROP QUOTA IF EXISTS qA, qB on instance test_quota/test.py::test_dcl_introspection Executing query DROP TABLE test.partition_complex on instance [gw5] PASSED test_partition/test.py::test_partition_complex Executing query INSERT INTO tbl VALUES (2, 'str2') on node2 test_partition/test.py::test_partition_simple Executing query DROP TABLE IF EXISTS test.partition_simple on instance thread 0: insert for 2001-01-01: [10, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 1: insert for 2003-01-02: [8, 7, 3, 7, 3, 2, 8, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query CREATE TABLE test.partition_simple (date MATERIALIZED toDate(0), x UInt64, sample_key MATERIALIZED intHash64(x)) ENGINE=MergeTree PARTITION BY date SAMPLE BY sample_key ORDER BY (date,x,sample_key) SETTINGS index_granularity=8192, index_granularity_bytes=0, compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1 on instance thread 2: insert for 2009-01-03: [5, 8, 3, 4, 9, 1, 6, 1, 7, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM SYNC REPLICA ON CLUSTER 'cluster' tbl on node1 Executing query SYSTEM RELOAD CONFIG on instance No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/8bfe5641f424a0d458743a009c7ecfe4a9e7078ddcdeea108436a5e408f02274/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/8bfe5641f424a0d458743a009c7ecfe4a9e7078ddcdeea108436a5e408f02274/json HTTP/1.1" 200 586 Executing query INSERT INTO test.partition_simple ( x ) VALUES ( now() ) on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 4: insert for 2004-01-05: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query INSERT INTO test.partition_simple ( x ) VALUES ( now()+1 ) on instance Executing query SYSTEM RELOAD CONFIG on instance [gw3] PASSED test_rename_column/test.py::test_rename_distributed test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select Executing query CREATE TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/{shard}/test_rename_distributed_parallel_insert_and_select_replicated', '{replica}') ORDER BY num PARTITION BY num % 100; on node1 Executing query SELECT * FROM tbl ORDER BY id on node1 Executing query ALTER TABLE test.partition_simple DETACH PARTITION 197001 on instance Executing query SHOW QUOTAS on instance Executing query SELECT * FROM tbl ORDER BY id on node2 Executing query ALTER TABLE test.partition_simple ATTACH PARTITION 197001 on instance Executing query SHOW CREATE QUOTA on instance thread 3: insert for 2001-01-04: [3, 9, 2, 1, 8, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 1: insert for 2004-01-02: [3, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 0: insert for 2004-01-01: [8, 10, 8, 1, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query CREATE TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster AS test_rename_distributed_parallel_insert_and_select_replicated ENGINE = Distributed(test_cluster, default, test_rename_distributed_parallel_insert_and_select_replicated, rand()) on node1 Executing query CHECK TABLE tbl on node1 Executing query OPTIMIZE TABLE test.partition_simple on instance Executing query SHOW CREATE QUOTAS on instance run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 4: insert for 2003-01-05: [1, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:73621 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SHOW QUOTA on instance Executing query DROP TABLE test.partition_simple on instance [gw5] PASSED test_partition/test.py::test_partition_simple thread 2: insert for 2009-01-03: [2, 6, 8, 1, 2, 4, 4, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query CHECK TABLE tbl on node2 Stdout:98217082 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(1000) on node1 Executing query SELECT * from test_table on instance test_partition/test.py::test_system_detached_parts Executing query SYSTEM STOP MERGES on instance Executing query OPTIMIZE TABLE tbl ON CLUSTER 'cluster' FINAL on node1 Stdout:98860056 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:69203 Executing query SHOW QUOTA on instance Executing query DROP TABLE IF EXISTS test.drop_detached on instance Executing query CREATE TABLE test.drop_detached (n UInt64) ENGINE = MergeTree() PARTITION BY intDiv(n, 8) ORDER BY n SETTINGS compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1 on instance Executing query SYSTEM SYNC REPLICA ON CLUSTER 'cluster' tbl on node1 Executing query SHOW ACCESS on instance thread 0: insert for 2009-01-01: [2, 7, 8, 6, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO test.drop_detached SELECT number FROM system.numbers WHERE number % 2 = 0 LIMIT 8 on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxkdXJhdGlvbj4zMTU1Njk1MjwvZHVyYXRpb24+CiAgICAgICAgICAgICAgICA8cXVlcmllcz4xMDAwPC9xdWVyaWVzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CgogICAgICAgICAgICA8aW50ZXJ2YWwyPgogICAgICAgICAgICAgICAgPHJhbmRvbWl6ZT50cnVlPC9yYW5kb21pemU+CiAgICAgICAgICAgICAgICA8ZHVyYXRpb24+NjMxMTM5MDQ8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPHJlYWRfYnl0ZXM+MjAwMDA8L3JlYWRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8cmVzdWx0X2J5dGVzPjMwMDAwPC9yZXN1bHRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8ZXhlY3V0aW9uX3RpbWU+MTIwPC9leGVjdXRpb25fdGltZT4KICAgICAgICAgICAgPC9pbnRlcnZhbDI+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxkdXJhdGlvbj4zMTU1Njk1MjwvZHVyYXRpb24+CiAgICAgICAgICAgICAgICA8cXVlcmllcz4xMDAwPC9xdWVyaWVzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CgogICAgICAgICAgICA8aW50ZXJ2YWwyPgogICAgICAgICAgICAgICAgPHJhbmRvbWl6ZT50cnVlPC9yYW5kb21pemU+CiAgICAgICAgICAgICAgICA8ZHVyYXRpb24+NjMxMTM5MDQ8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPHJlYWRfYnl0ZXM+MjAwMDA8L3JlYWRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8cmVzdWx0X2J5dGVzPjMwMDAwPC9yZXN1bHRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8ZXhlY3V0aW9uX3RpbWU+MTIwPC9leGVjdXRpb25fdGltZT4KICAgICAgICAgICAgPC9pbnRlcnZhbDI+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 2: insert for 2009-01-03: [7, 10, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query INSERT INTO test.drop_detached SELECT number FROM system.numbers WHERE number % 2 = 1 LIMIT 8 on instance Executing query SYSTEM RELOAD CONFIG on instance Executing query SELECT * FROM tbl ORDER BY id on node1 thread 3: insert for 2002-01-04: [8, 5, 7, 5, 7, 3, 9, 9, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2010-01-05: [2, 3, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 1: insert for 2003-01-02: [5, 1, 10, 5, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query drop table if exists sdp_0 sync on instance Executing query SHOW QUOTAS on instance Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query SELECT * FROM tbl ORDER BY id on node2 Executing query SHOW CREATE QUOTA on instance Executing query drop table if exists sdp_1 sync on instance Executing query CHECK TABLE tbl on node1 Executing query drop table if exists sdp_2 sync on instance Executing query SHOW QUOTA on instance Executing query CHECK TABLE tbl on node2 thread 0: insert for 2005-01-01: [3, 10, 8, 6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query drop table if exists sdp_3 sync on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxkdXJhdGlvbj4zMTU1Njk1MjwvZHVyYXRpb24+CiAgICAgICAgICAgICAgICA8cXVlcmllcz4xMDAwPC9xdWVyaWVzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgoKICAgICAgICA8bXlRdW90YTI+CiAgICAgICAgICAgIDxrZXllZC8+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxyYW5kb21pemU+dHJ1ZTwvcmFuZG9taXplPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjM2MDA8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz40MDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+NDAwMDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVhZF9ieXRlcz40MDAwMDA8L3JlYWRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8cmVzdWx0X2J5dGVzPjQwMDAwMDwvcmVzdWx0X2J5dGVzPgogICAgICAgICAgICAgICAgPGV4ZWN1dGlvbl90aW1lPjYwPC9leGVjdXRpb25fdGltZT4KICAgICAgICAgICAgICAgIDxmYWlsZWRfc2VxdWVudGlhbF9hdXRoZW50aWNhdGlvbnM+MzwvZmFpbGVkX3NlcXVlbnRpYWxfYXV0aGVudGljYXRpb25zPgogICAgICAgICAgICA8L2ludGVydmFsPgogICAgICAgICAgICA8aW50ZXJ2YWwyPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjI2Mjk3NDY8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPGV4ZWN1dGlvbl90aW1lPjE4MDA8L2V4ZWN1dGlvbl90aW1lPgogICAgICAgICAgICA8L2ludGVydmFsMj4KICAgICAgICA8L215UXVvdGEyPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxkdXJhdGlvbj4zMTU1Njk1MjwvZHVyYXRpb24+CiAgICAgICAgICAgICAgICA8cXVlcmllcz4xMDAwPC9xdWVyaWVzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgoKICAgICAgICA8bXlRdW90YTI+CiAgICAgICAgICAgIDxrZXllZC8+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDxyYW5kb21pemU+dHJ1ZTwvcmFuZG9taXplPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjM2MDA8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz40MDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+NDAwMDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVhZF9ieXRlcz40MDAwMDA8L3JlYWRfYnl0ZXM+CiAgICAgICAgICAgICAgICA8cmVzdWx0X2J5dGVzPjQwMDAwMDwvcmVzdWx0X2J5dGVzPgogICAgICAgICAgICAgICAgPGV4ZWN1dGlvbl90aW1lPjYwPC9leGVjdXRpb25fdGltZT4KICAgICAgICAgICAgICAgIDxmYWlsZWRfc2VxdWVudGlhbF9hdXRoZW50aWNhdGlvbnM+MzwvZmFpbGVkX3NlcXVlbnRpYWxfYXV0aGVudGljYXRpb25zPgogICAgICAgICAgICA8L2ludGVydmFsPgogICAgICAgICAgICA8aW50ZXJ2YWwyPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjI2Mjk3NDY8L2R1cmF0aW9uPgogICAgICAgICAgICAgICAgPGV4ZWN1dGlvbl90aW1lPjE4MDA8L2V4ZWN1dGlvbl90aW1lPgogICAgICAgICAgICA8L2ludGVydmFsMj4KICAgICAgICA8L215UXVvdGEyPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/.env --project-name roottestreplicatedmergetreeencryptioncodec-gw7 --file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node2/docker-compose.yml stop --timeout 20] [gw7] PASSED test_replicated_merge_tree_encryption_codec/test.py::test_same_keys Executing query create table sdp_0 (n int, x int) engine=MergeTree order by n SETTINGS compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1 on instance Executing query SYSTEM RELOAD CONFIG on instance run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 2: insert for 2003-01-03: [4, 2, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 3: insert for 2006-01-04: [1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:75667 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query create table sdp_1 (n int, x int) engine=MergeTree order by n partition by x SETTINGS compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1 on instance run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SHOW QUOTAS on instance thread 11: delete 26 * 9 Executing query ALTER TABLE test_mutations DELETE WHERE x = 9 on node1 Stdout:105278070 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 1: insert for 2005-01-02: [2, 4, 6, 2, 6, 2, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:1495 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 Executing query create table sdp_2 (n int, x String) engine=MergeTree order by n partition by x SETTINGS compress_marks=false, compress_primary_key=false, ratio_of_defaults_for_sparse_serialization=1 on instance Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(10) on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(10) on node2 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(10) on node3 Executing query SHOW CREATE QUOTA myQuota on instance Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo2 % 1000 > 0 on node1 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo3 % 1000 > 0 on node2 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 Stdout:105862116 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 4: insert for 2002-01-05: [6, 1, 6, 9, 9, 7, 7, 10, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo3 % 1000 > 0 on node2 thread 0: insert for 2007-01-01: [1, 2, 3, 9, 6, 3, 4, 2, 2, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:1495 Executing query select 20 on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(10) on node3 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(10) on node1 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo2 % 1000 > 0 on node1 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo3 % 1000 > 0 on node2 Executing query create table sdp_3 (n int, x Enum('broken' = 0, 'all' = 1)) engine=MergeTree order by n partition by x on instance Stdout:71513 Executing query SHOW CREATE QUOTA myQuota2 on instance Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo3 % 1000 > 0 on node2 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(10) on node3 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(10) on node1 thread 13: delete 41 * 5 Executing query ALTER TABLE test_mutations DELETE WHERE x = 5 on node1 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo2 % 1000 > 0 on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(10) on node2 Executing query system stop merges sdp_0 on instance Executing query SHOW CREATE QUOTAS on instance Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo3 % 1000 > 0 on node2 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(10) on node1 thread 3: insert for 2008-01-04: [4, 4, 6, 7, 4, 10, 6, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(10) on node3 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo2 % 1000 > 0 on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(10) on node2 Executing query insert into sdp_0 values (0, 0) on instance Executing query SHOW QUOTA on instance Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE foo2 % 1000 > 0 on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(10) on node3 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(10) on node1 thread 1: insert for 2001-01-02: [3, 9, 10, 6, 2, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(10) on node2 Executing query insert into sdp_0 values (1, 1) on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 thread 2: insert for 2000-01-03: [7, 10, 8, 10, 4, 1, 5, 4, 10, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(10) on node2 Executing query SYSTEM RELOAD CONFIG on instance Executing query select distinct partition_id from system.parts where table='sdp_0' on instance Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 thread 12: delete 46 * 2 Executing query ALTER TABLE test_mutations DELETE WHERE x = 2 on node2 thread 4: insert for 2008-01-05: [9, 4, 10, 10, 3, 4, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SHOW QUOTAS on instance Executing query alter table sdp_0 detach partition id 'all' on instance Executing query SHOW CREATE QUOTA on instance thread 0: insert for 2006-01-01: [2, 5, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query system stop merges sdp_1 on instance Executing query SHOW QUOTA on instance Executing query insert into sdp_1 values (0, 0) on instance thread 2: insert for 2000-01-03: [9, 3, 8, 7, 10, 6, 8, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query DROP QUOTA IF EXISTS qA, qB on instance [gw1] PASSED test_quota/test.py::test_dcl_introspection test_quota/test.py::test_dcl_management Executing query insert into sdp_1 values (1, 1) on instance thread 3: insert for 2008-01-04: [2, 8, 9, 10, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Stdout:77845 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SYSTEM RELOAD CONFIG on instance Executing query select distinct partition_id from system.parts where table='sdp_1' on instance thread 4: insert for 2003-01-05: [2, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:112970414 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 1: insert for 2007-01-02: [3, 10, 5, 7, 10, 9, 4, 2, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query alter table sdp_1 detach partition id '0' on instance Stdout:113477842 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SYSTEM RELOAD CONFIG on instance Stdout:73625 Executing query alter table sdp_1 detach partition id '1' on instance thread 0: insert for 2007-01-01: [8, 7, 8, 1, 9, 6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query system stop merges sdp_2 on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 2: insert for 2010-01-03: [3, 2, 10, 3, 4, 5, 3, 3, 9, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM RELOAD CONFIG on instance Executing query insert into sdp_2 values (0, 0) on instance Executing query SHOW QUOTA on instance Executing query insert into sdp_2 values (1, 1) on instance thread 3: insert for 2001-01-04: [2, 3, 3, 6, 10, 8, 6, 1, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query CREATE QUOTA qA FOR INTERVAL 15 MONTH MAX QUERIES 123 TO CURRENT_USER on instance Executing query select distinct partition_id from system.parts where table='sdp_2' on instance Executing query SHOW CREATE QUOTA qA on instance thread 1: insert for 2003-01-02: [3, 5, 5, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2008-01-05: [2, 10, 10, 5, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query alter table sdp_2 detach partition id '58ed7160db50ea45e1c6aa694c8cbfd1' on instance Executing query SHOW QUOTA on instance thread 2: insert for 2005-01-03: [7, 3, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2002-01-01: [7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query alter table sdp_2 detach partition id '6711e2b2592d86d18fc0f260cf33ef2b' on instance Executing query SELECT * from test_table on instance Executing query system stop merges sdp_3 on instance Executing query SHOW QUOTA on instance Executing query insert into sdp_3 values (0, 0) on instance Executing query DROP TABLE IF EXISTS limited_fetch_table SYNC on node1 thread 3: insert for 2009-01-04: [9, 1, 8, 10, 4, 4, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query ALTER QUOTA qA FOR INTERVAL 15 MONTH MAX QUERIES 321, MAX ERRORS 10, FOR INTERVAL 0.5 HOUR MAX EXECUTION TIME 0.5 on instance Executing query insert into sdp_3 values (1, 1) on instance Executing query SHOW CREATE QUOTA qA on instance Executing query select distinct partition_id from system.parts where table='sdp_3' on instance Executing query SHOW QUOTA on instance Executing query alter table sdp_3 detach partition id '0' on instance Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 Executing query SELECT * from test_table on instance Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 thread 1: insert for 2007-01-02: [8, 3, 9, 10, 7, 6, 5, 10, 9, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query alter table sdp_3 detach partition id '1' on instance Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 thread 4: insert for 2007-01-05: [7, 3, 8, 2, 1, 1, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2002-01-03: [8, 5, 6, 10, 7, 2, 5, 2, 6, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 thread 0: insert for 2003-01-01: [1, 6, 5, 4, 5, 1, 5, 8, 8, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SHOW QUOTA on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_0/detached/attaching_0_6_6_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_0/detached/attaching_0_6_6_0] Executing query ALTER QUOTA qA FOR INTERVAL 15 MONTH NO LIMITS, FOR RANDOMIZED INTERVAL 16 MONTH TRACKING ONLY, FOR INTERVAL 1800 SECOND NO LIMITS on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_0/detached/deleting_0_7_7_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_0/detached/deleting_0_7_7_0] Executing query SHOW QUOTA on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_0/detached/any_other_name'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_0/detached/any_other_name] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_0/detached/prefix_1_2_2_0_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_0/detached/prefix_1_2_2_0_0] Executing query SELECT * from test_table on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_0/detached/ignored_202107_714380_714380_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_0/detached/ignored_202107_714380_714380_0] thread 3: insert for 2007-01-04: [6, 3, 2, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_0/detached/broken_202107_714380_714380_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_0/detached/broken_202107_714380_714380_123] Executing query SHOW QUOTA on instance thread 4: insert for 2000-01-05: [3, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Command:[docker compose --env-file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/.env --project-name roottestpostgresqlreplicadatabaseengine1-gw0 --file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_postgres.yml pause postgres1] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_0/detached/clone_all_714380_714380_42'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_0/detached/clone_all_714380_714380_42] Executing query ALTER QUOTA qA RENAME TO qB on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_0/detached/clone_all_714380_714380_42_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_0/detached/clone_all_714380_714380_42_123] thread 1: insert for 2009-01-02: [8, 6, 10, 9, 10, 5, 1, 8, 6, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2004-01-01: [4, 4, 2, 6, 2, 5, 3, 8, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_0/detached/broken-on-start_6711e2b2592d86d18fc0f260cf33ef2b_714380_714380_42_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_0/detached/broken-on-start_6711e2b2592d86d18fc0f260cf33ef2b_714380_714380_42_123] Executing query SHOW CREATE QUOTA qB on instance Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Paused Command:[docker compose --env-file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/.env --project-name roottestpostgresqlreplicadatabaseengine1-gw0 --file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_postgres.yml unpause postgres1] thread 2: insert for 2004-01-03: [8, 6, 5, 2, 10, 6, 3, 10, 4, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_1/detached/attaching_0_6_6_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_1/detached/attaching_0_6_6_0] Executing query SHOW QUOTA on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_1/detached/deleting_0_7_7_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_1/detached/deleting_0_7_7_0] Executing query DROP TABLE IF EXISTS limited_fetch_table SYNC on node2 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_1/detached/any_other_name'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_1/detached/any_other_name] Executing query SELECT * from test_table on instance Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Unpaused Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance thread 13: delete 34 * 8 Executing query ALTER TABLE test_mutations DELETE WHERE x = 8 on node2 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_1/detached/prefix_1_2_2_0_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_1/detached/prefix_1_2_2_0_0] thread 11: delete 55 * 10 Executing query ALTER TABLE test_mutations DELETE WHERE x = 10 on node2 Executing query SHOW QUOTA on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_1/detached/ignored_202107_714380_714380_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_1/detached/ignored_202107_714380_714380_0] Executing query CREATE TABLE test_add_disk_to_policy ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_1/detached/broken_202107_714380_714380_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_1/detached/broken_202107_714380_714380_123] Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Executing query DROP QUOTA qB on instance thread 3: insert for 2004-01-04: [8, 8, 6, 4, 2, 5, 2, 8, 4, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SYSTEM RELOAD CONFIG on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_1/detached/clone_all_714380_714380_42'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_1/detached/clone_all_714380_714380_42] thread 12: delete 25 * 1 Executing query ALTER TABLE test_mutations DELETE WHERE x = 1 on node2 thread 4: insert for 2005-01-05: [5, 8, 7, 10, 6, 2, 2, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SHOW QUOTA on instance Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_1/detached/clone_all_714380_714380_42_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_1/detached/clone_all_714380_714380_42_123] thread 2: insert for 2001-01-03: [9, 1, 5, 9, 4, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_1/detached/broken-on-start_6711e2b2592d86d18fc0f260cf33ef2b_714380_714380_42_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_1/detached/broken-on-start_6711e2b2592d86d18fc0f260cf33ef2b_714380_714380_42_123] thread 1: insert for 2001-01-02: [7, 9, 7, 2, 10, 6, 6, 1, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query DROP QUOTA IF EXISTS qA, qB on instance [gw1] PASSED test_quota/test.py::test_dcl_management test_quota/test.py::test_exceed_quota Executing query SELECT volume_name FROM system.storage_policies WHERE policy_name = 'cool_policy' on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_2/detached/attaching_0_6_6_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_2/detached/attaching_0_6_6_0] Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance thread 0: insert for 2008-01-01: [1, 3, 2, 9, 6, 7, 1, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_2/detached/deleting_0_7_7_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_2/detached/deleting_0_7_7_0] Executing query SELECT disks FROM system.storage_policies WHERE policy_name = 'cool_policy' on node1 Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_2/detached/any_other_name'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_2/detached/any_other_name] Executing query SYSTEM RELOAD CONFIG on instance Executing query DROP TABLE IF EXISTS test_add_disk_to_policy on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_2/detached/prefix_1_2_2_0_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_2/detached/prefix_1_2_2_0_0] Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance thread 3: insert for 2002-01-04: [10, 3, 1, 3, 5, 7, 6, 4, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_2/detached/ignored_202107_714380_714380_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_2/detached/ignored_202107_714380_714380_0] [gw2] PASSED test_reloading_storage_configuration/test.py::test_add_disk_to_policy test_reloading_storage_configuration/test.py::test_add_disk_to_separate_config run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] Executing query SYSTEM RELOAD CONFIG on instance Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_2/detached/broken_202107_714380_714380_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_2/detached/broken_202107_714380_714380_123] Executing query CREATE TABLE limited_fetches0(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetches0', '0') ORDER BY tuple() PARTITION BY key on node1 [gw6] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server thread 4: insert for 2002-01-05: [8, 8, 9, 7] Stdout: PID TTY TIME CMD thread 1: insert for 2010-01-02: [5, 5, 4, 5, 10, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIFRpbnkgbGltaXRzLiAtLT4KICAgICAgICAgICAgICAgIDxxdWVyaWVzPjE8L3F1ZXJpZXM+CiAgICAgICAgICAgICAgICA8cXVlcnlfc2VsZWN0cz4xPC9xdWVyeV9zZWxlY3RzPgogICAgICAgICAgICAgICAgPHF1ZXJ5X2luc2VydHM+MTwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MTwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MTwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgICAgICA8ZmFpbGVkX3NlcXVlbnRpYWxfYXV0aGVudGljYXRpb25zPjE8L2ZhaWxlZF9zZXF1ZW50aWFsX2F1dGhlbnRpY2F0aW9ucz4KICAgICAgICAgICAgPC9pbnRlcnZhbD4KICAgICAgICA8L215UXVvdGE+CiAgICA8L3F1b3Rhcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIFRpbnkgbGltaXRzLiAtLT4KICAgICAgICAgICAgICAgIDxxdWVyaWVzPjE8L3F1ZXJpZXM+CiAgICAgICAgICAgICAgICA8cXVlcnlfc2VsZWN0cz4xPC9xdWVyeV9zZWxlY3RzPgogICAgICAgICAgICAgICAgPHF1ZXJ5X2luc2VydHM+MTwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MTwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MTwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgICAgICA8ZmFpbGVkX3NlcXVlbnRpYWxfYXV0aGVudGljYXRpb25zPjE8L2ZhaWxlZF9zZXF1ZW50aWFsX2F1dGhlbnRpY2F0aW9ucz4KICAgICAgICAgICAgPC9pbnRlcnZhbD4KICAgICAgICA8L215UXVvdGE+CiAgICA8L3F1b3Rhcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Stdout: 1495 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] thread 2: insert for 2006-01-03: [3, 4, 8, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_2/detached/clone_all_714380_714380_42'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_2/detached/clone_all_714380_714380_42] Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance thread 0: insert for 2000-01-01: [1, 10, 6, 3, 7, 4, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SYSTEM RELOAD CONFIG on instance run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query CREATE TABLE limited_fetches1(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetches1', '0') ORDER BY tuple() PARTITION BY key on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_2/detached/clone_all_714380_714380_42_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_2/detached/clone_all_714380_714380_42_123] Stdout:1495 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_2/detached/broken-on-start_6711e2b2592d86d18fc0f260cf33ef2b_714380_714380_42_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_2/detached/broken-on-start_6711e2b2592d86d18fc0f260cf33ef2b_714380_714380_42_123] Executing query SELECT * FROM system.quotas ORDER BY name on instance Executing query CREATE TABLE limited_fetches2(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetches2', '0') ORDER BY tuple() PARTITION BY key on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_3/detached/attaching_0_6_6_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_3/detached/attaching_0_6_6_0] Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 thread 3: insert for 2009-01-04: [4, 2, 7, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_3/detached/deleting_0_7_7_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_3/detached/deleting_0_7_7_0] Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo2 to foo3 on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_3/detached/any_other_name'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_3/detached/any_other_name] Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Executing query CREATE TABLE limited_fetches3(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetches3', '0') ORDER BY tuple() PARTITION BY key on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_3/detached/prefix_1_2_2_0_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_3/detached/prefix_1_2_2_0_0] Executing query SELECT * from test_table on instance Executing query CREATE TABLE limited_fetches4(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetches4', '0') ORDER BY tuple() PARTITION BY key on node1 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_3/detached/ignored_202107_714380_714380_0'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_3/detached/ignored_202107_714380_714380_0] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_3/detached/broken_202107_714380_714380_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_3/detached/broken_202107_714380_714380_123] Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance thread 2: insert for 2009-01-03: [7, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_3/detached/clone_all_714380_714380_42'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_3/detached/clone_all_714380_714380_42] thread 4: insert for 2006-01-05: [6, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query CREATE TABLE limited_fetches0(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetches0', '1') ORDER BY tuple() PARTITION BY key on node3 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_3/detached/clone_all_714380_714380_42_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_3/detached/clone_all_714380_714380_42_123] thread 1: insert for 2008-01-02: [6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2003-01-01: [8, 10, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 3: insert for 2006-01-04: [7, 7, 1, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM RELOAD CONFIG on instance Executing query CREATE TABLE limited_fetches1(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetches1', '1') ORDER BY tuple() PARTITION BY key on node3 run container_id:roottestpartition-gw5-instance-1 detach:False nothrow:False cmd: ['mkdir', '/var/lib/clickhouse/data/default/sdp_3/detached/broken-on-start_6711e2b2592d86d18fc0f260cf33ef2b_714380_714380_42_123'] Command:[docker exec roottestpartition-gw5-instance-1 mkdir /var/lib/clickhouse/data/default/sdp_3/detached/broken-on-start_6711e2b2592d86d18fc0f260cf33ef2b_714380_714380_42_123] Executing query SELECT * FROM system.quotas ORDER BY name on instance Executing query select system.detached_parts.* except (bytes_on_disk, `path`, modification_time) from system.detached_parts where table like 'sdp_%' order by table, name on instance Executing query CREATE TABLE limited_fetches2(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetches2', '1') ORDER BY tuple() PARTITION BY key on node3 Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Executing query select distinct partition_id from system.detached_parts where table='sdp_0' and partition_id is not null on instance Executing query CREATE TABLE limited_fetches3(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetches3', '1') ORDER BY tuple() PARTITION BY key on node3 Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Executing query alter table sdp_0 attach partition id '202107' on instance Executing query CREATE TABLE limited_fetches4(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetches4', '1') ORDER BY tuple() PARTITION BY key on node3 Executing query SELECT * from test_table on instance Executing query alter table sdp_0 attach partition id 'all' on instance Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance thread 0: insert for 2009-01-01: [1, 2, 10, 3, 3, 6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query alter table sdp_0 attach partition id '6711e2b2592d86d18fc0f260cf33ef2b' on instance thread 3: insert for 2009-01-04: [7, 1, 7, 7, 4, 8, 10, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2009-01-05: [6, 6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 1: insert for 2002-01-02: [9, 3, 1, 9, 3, 10, 6, 7, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM STOP FETCHES limited_fetches0 on node3 thread 2: insert for 2010-01-03: [6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 [gw1] PASSED test_quota/test.py::test_exceed_quota Executing query alter table sdp_0 attach partition id '0' on instance Executing query DROP QUOTA IF EXISTS qA, qB on instance test_quota/test.py::test_query_inserts run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query INSERT INTO limited_fetches0 SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query alter table sdp_0 attach partition id '1' on instance Executing query SYSTEM RELOAD CONFIG on instance Executing query select distinct partition_id from system.detached_parts where table='sdp_1' and partition_id is not null on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query INSERT INTO limited_fetches0 SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query alter table sdp_1 attach partition id '202107' on instance thread 0: insert for 2002-01-01: [3, 2, 6, 5, 3, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SYSTEM RELOAD CONFIG on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Executing query alter table sdp_1 attach partition id '0' on instance run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query alter table sdp_1 attach partition id 'all' on instance No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 Executing query INSERT INTO limited_fetches0 SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query SELECT * FROM system.quotas ORDER BY name on instance http://localhost:None "POST /v1.46/exec/2788f850cadc587bc74f1f5e46d076ea9f8968b4f420bf6d7075bc3d2c125c55/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/2788f850cadc587bc74f1f5e46d076ea9f8968b4f420bf6d7075bc3d2c125c55/json HTTP/1.1" 200 586 thread 2: insert for 2007-01-03: [7, 5, 6, 4, 10, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2001-01-04: [7, 3, 8, 9, 1, 8, 8, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query alter table sdp_1 attach partition id '1' on instance thread 1: insert for 2008-01-02: [1, 9, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance thread 4: insert for 2001-01-05: [6, 1, 10, 2, 6, 6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 11: delete 62 * 7 Executing query ALTER TABLE test_mutations DELETE WHERE x = 7 on node1 Executing query alter table sdp_1 attach partition id '6711e2b2592d86d18fc0f260cf33ef2b' on instance Executing query INSERT INTO limited_fetches0 SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Executing query select distinct partition_id from system.detached_parts where table='sdp_2' and partition_id is not null on instance Executing query SELECT quota_name, quota_key, is_current, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows, result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quotas_usage ORDER BY quota_name, quota_key, duration on instance Executing query INSERT INTO limited_fetches0 SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query alter table sdp_2 attach partition id '202107' on instance Executing query DROP TABLE IF EXISTS test_table_ins on instance Executing query alter table sdp_2 attach partition id 'all' on instance Executing query CREATE TABLE test_table_ins(x UInt32) ENGINE = MergeTree ORDER BY tuple() on instance Executing query alter table sdp_2 attach partition id '6711e2b2592d86d18fc0f260cf33ef2b' on instance Executing query SYSTEM STOP FETCHES limited_fetches1 on node3 thread 0: insert for 2009-01-01: [2, 2, 7, 8, 8, 1, 7, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance thread 4: insert for 2007-01-05: [8, 7, 7, 5, 1, 5, 2, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 Executing query alter table sdp_2 attach partition id '0' on instance Executing query INSERT INTO limited_fetches1 SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query INSERT INTO test_table_ins values(1) on instance thread 1: insert for 2002-01-02: [10, 10, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 3: insert for 2009-01-04: [8, 2, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 2: insert for 2000-01-03: [9, 7, 10, 6, 9, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query alter table sdp_2 attach partition id '58ed7160db50ea45e1c6aa694c8cbfd1' on instance Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Executing query INSERT INTO limited_fetches1 SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query alter table sdp_2 attach partition id '1' on instance Executing query DROP TABLE test_table_ins on instance Executing query select distinct partition_id from system.detached_parts where table='sdp_3' and partition_id is not null on instance [gw1] PASSED test_quota/test.py::test_query_inserts test_quota/test.py::test_quota_from_users_xml Executing query DROP QUOTA IF EXISTS qA, qB on instance run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query INSERT INTO limited_fetches1 SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 13: delete 43 * 4 Executing query ALTER TABLE test_mutations DELETE WHERE x = 4 on node1 Executing query alter table sdp_3 attach partition id '202107' on instance Executing query SYSTEM RELOAD CONFIG on instance Executing query alter table sdp_3 attach partition id '0' on instance thread 4: insert for 2006-01-05: [2, 4, 9, 2, 10, 8, 7, 4, 7, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query INSERT INTO limited_fetches1 SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 3: insert for 2004-01-04: [5, 10, 1, 4, 8, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 1: insert for 2007-01-02: [2, 2, 9, 5, 1, 10, 4, 1, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 0: insert for 2000-01-01: [2, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query alter table sdp_3 attach partition id 'all' on instance Executing query SYSTEM RELOAD CONFIG on instance Executing query alter table sdp_3 attach partition id '1' on instance Executing query INSERT INTO limited_fetches1 SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query alter table sdp_3 attach partition id '6711e2b2592d86d18fc0f260cf33ef2b' on instance Executing query SELECT * FROM system.quotas ORDER BY name on instance thread 2: insert for 2010-01-03: [9, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query select n, x::int AS x, count() from merge('default', '^sdp_') group by n, x on instance Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Executing query SYSTEM STOP FETCHES limited_fetches2 on node3 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance thread 1: insert for 2002-01-02: [4, 8, 2, 8, 10, 7, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO limited_fetches2 SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Stdout:2255 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query DROP TABLE test.drop_detached on instance [gw5] PASSED test_partition/test.py::test_system_detached_parts Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query SELECT quota_name, quota_key, is_current, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows, result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quotas_usage ORDER BY quota_name, quota_key, duration on instance Stdout:2255 Executing query select 20 on node1 Executing query SYSTEM START MERGES on instance thread 3: insert for 2007-01-04: [4, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT * from test_table on instance thread 4: insert for 2010-01-05: [5, 3, 5, 2, 10, 1, 7, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2001-01-01: [4, 7, 3, 5, 8, 8, 2, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO limited_fetches2 SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Command:[docker compose --env-file /ClickHouse/tests/integration/test_partition/_instances-0-gw5/.env --project-name roottestpartition-gw5 --file /ClickHouse/tests/integration/test_partition/_instances-0-gw5/instance/docker-compose.yml stop --timeout 20] Stderr: zoo1 Skipped - Image is already being pulled by zoo3 Stderr: node1 Skipped - Image is already being pulled by node2 Stderr: zoo2 Skipped - Image is already being pulled by zoo3 Stderr: node2 Pulling Stderr: zoo3 Pulling Stderr: node2 Pulled Stderr: zoo3 Pulled Setup ZooKeeper Secure Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk1/data', '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk1/log', '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk2/data', '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk2/log', '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk3/data', '/ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/zk3/log'] Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml --verbose up -d] Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Executing query SELECT SUM(x) from test_table on instance Executing query INSERT INTO limited_fetches2 SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 12: delete 66 * 3 Executing query ALTER TABLE test_mutations DELETE WHERE x = 3 on node1 thread 2: insert for 2001-01-03: [9, 10, 9, 7, 4, 8, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 1: insert for 2007-01-02: [10, 2, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Executing query INSERT INTO limited_fetches2 SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 4: insert for 2009-01-05: [7, 2, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 [gw1] PASSED test_quota/test.py::test_quota_from_users_xml Executing query DROP QUOTA IF EXISTS qA, qB on instance test_quota/test.py::test_reload_users_xml_by_timer thread 3: insert for 2004-01-04: [3, 5, 2, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 0: insert for 2003-01-01: [5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query INSERT INTO limited_fetches2 SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query SYSTEM RELOAD CONFIG on instance thread 2: insert for 2009-01-03: [7, 10, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM STOP FETCHES limited_fetches3 on node3 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 11: delete 44 * 5 Executing query ALTER TABLE test_mutations DELETE WHERE x = 5 on node1 Executing query INSERT INTO limited_fetches3 SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 4: insert for 2008-01-05: [6, 4, 2, 7, 9, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 1: insert for 2002-01-02: [6, 7, 6, 6, 10, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM RELOAD CONFIG on instance Executing query INSERT INTO limited_fetches3 SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query SELECT * FROM system.quotas ORDER BY name on instance thread 3: insert for 2010-01-04: [8, 3, 1, 1, 1, 7, 7, 7, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2000-01-03: [3, 2, 10, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance thread 0: insert for 2001-01-01: [6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO limited_fetches3 SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN num2 to foo2 on node1 thread 1: insert for 2000-01-02: [9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO limited_fetches3 SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 4: insert for 2007-01-05: [2, 3, 4, 1, 3, 5, 9, 1, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query INSERT INTO limited_fetches3 SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 3: insert for 2003-01-04: [9, 1, 3, 6, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2006-01-03: [8, 1, 2, 5, 1, 4, 4, 8, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2007-01-01: [1, 7, 7, 9, 9, 3, 6, 6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SYSTEM STOP FETCHES limited_fetches4 on node3 Stderr:time="2025-04-02T03:31:55Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreloadclientcertificate-gw4_default Creating Stderr: Network roottestreloadclientcertificate-gw4_default Created Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Creating Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Creating Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Creating Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Created Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Created Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Created Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Started Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Started Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Started Stderr:time="2025-04-02T03:31:56Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:31:56Z" level=debug msg="otel error" error="" Wait ZooKeeper Secure to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.2.3, port:2281, use_ssl:True Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True Connection dropped: socket connection error: Connection refused thread 13: delete 26 * 7 Executing query ALTER TABLE test_mutations DELETE WHERE x = 7 on node1 thread 1: insert for 2004-01-02: [4, 9, 8, 1, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO limited_fetches4 SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True Connection dropped: socket connection error: Connection refused thread 4: insert for 2009-01-05: [9, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query INSERT INTO limited_fetches4 SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 12: delete 36 * 1 Executing query ALTER TABLE test_mutations DELETE WHERE x = 1 on node2 thread 3: insert for 2002-01-04: [7, 1, 1, 4, 7, 1, 9, 2, 8, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Executing query INSERT INTO limited_fetches4 SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 0: insert for 2008-01-01: [1, 9, 4, 5, 5, 7, 3, 1, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2000-01-03: [6, 8, 8, 9, 6, 3, 8, 6, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO limited_fetches4 SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 1: insert for 2004-01-02: [9, 9, 6, 9, 3, 1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIFRpbnkgbGltaXRzLiAtLT4KICAgICAgICAgICAgICAgIDxxdWVyaWVzPjE8L3F1ZXJpZXM+CiAgICAgICAgICAgICAgICA8cXVlcnlfc2VsZWN0cz4xPC9xdWVyeV9zZWxlY3RzPgogICAgICAgICAgICAgICAgPHF1ZXJ5X2luc2VydHM+MTwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MTwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MTwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgICAgICA8ZmFpbGVkX3NlcXVlbnRpYWxfYXV0aGVudGljYXRpb25zPjE8L2ZhaWxlZF9zZXF1ZW50aWFsX2F1dGhlbnRpY2F0aW9ucz4KICAgICAgICAgICAgPC9pbnRlcnZhbD4KICAgICAgICA8L215UXVvdGE+CiAgICA8L3F1b3Rhcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIFRpbnkgbGltaXRzLiAtLT4KICAgICAgICAgICAgICAgIDxxdWVyaWVzPjE8L3F1ZXJpZXM+CiAgICAgICAgICAgICAgICA8cXVlcnlfc2VsZWN0cz4xPC9xdWVyeV9zZWxlY3RzPgogICAgICAgICAgICAgICAgPHF1ZXJ5X2luc2VydHM+MTwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MTwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MTwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgICAgICA8ZmFpbGVkX3NlcXVlbnRpYWxfYXV0aGVudGljYXRpb25zPjE8L2ZhaWxlZF9zZXF1ZW50aWFsX2F1dGhlbnRpY2F0aW9ucz4KICAgICAgICAgICAgPC9pbnRlcnZhbD4KICAgICAgICA8L215UXVvdGE+CiAgICA8L3F1b3Rhcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query INSERT INTO limited_fetches4 SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query SELECT * FROM system.quotas on instance thread 3: insert for 2010-01-04: [5, 9, 10, 4, 8, 9, 3, 9, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 4: insert for 2008-01-05: [9, 2, 6, 10, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2008-01-01: [5, 9, 5, 3, 8, 4, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query CREATE TABLE test_add_disk ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 Executing query SELECT * FROM system.quota_limits on instance run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk \'{print $1 " " $2}\' /proc/net/route | grep 00000000 | awk \'{print $1}\''] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk '{print $1 " " $2}' /proc/net/route | grep 00000000 | awk '{print $1}'] Executing query SELECT name FROM system.disks on node1 Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True Connection dropped: socket connection error: Connection refused thread 11: delete 29 * 10 Executing query ALTER TABLE test_mutations DELETE WHERE x = 10 on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Stdout:eth0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c grep "^ *eth0:" /proc/net/dev] thread 1: insert for 2010-01-02: [8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SYSTEM RELOAD CONFIG on node1 Stdout: eth0: 236136 1918 0 0 0 0 0 0 130638695 7887 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query SELECT name FROM system.disks on node1 thread 2: insert for 2010-01-03: [4, 5, 3, 3, 2, 1, 7, 10, 10, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout: eth0: 236136 1918 0 0 0 0 0 0 130638695 7887 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:236136 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:130638695 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk \'{print $1 " " $2}\' /proc/net/route | grep 00000000 | awk \'{print $1}\''] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk '{print $1 " " $2}' /proc/net/route | grep 00000000 | awk '{print $1}'] Executing query DROP TABLE IF EXISTS test_add_disk on node1 thread 3: insert for 2006-01-04: [10, 1, 7, 3, 5, 9, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:eth0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c grep "^ *eth0:" /proc/net/dev] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] [gw2] PASSED test_reloading_storage_configuration/test.py::test_add_disk_to_separate_config test_reloading_storage_configuration/test.py::test_add_policy thread 4: insert for 2006-01-05: [1, 3, 9, 7, 4, 9, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 0: insert for 2010-01-01: [1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout: eth0: 105809 670 0 0 0 0 0 0 106481 665 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c grep "^ *eth0:" /proc/net/dev] thread 1: insert for 2009-01-02: [8, 10, 8, 4, 10, 10, 8, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout: PID TTY TIME CMD Stdout: 2255 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] Stdout: eth0: 105809 670 0 0 0 0 0 0 106481 665 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:105809 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SELECT * FROM system.quota_limits on instance thread 2: insert for 2007-01-03: [1, 10, 10, 1, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:2255 Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Stdout:106481 Executing query SYSTEM START FETCHES limited_fetches0 on node3 thread 3: insert for 2010-01-04: [9, 8, 10, 3, 10, 7, 2, 10, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 [gw1] PASSED test_quota/test.py::test_reload_users_xml_by_timer test_quota/test.py::test_simpliest_quota Executing query DROP QUOTA IF EXISTS qA, qB on instance Executing query SYSTEM START FETCHES limited_fetches1 on node3 Executing query SYSTEM START FETCHES limited_fetches2 on node3 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query SYSTEM START FETCHES limited_fetches3 on node3 thread 4: insert for 2003-01-05: [9, 7, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SYSTEM RELOAD CONFIG on instance Executing query SYSTEM START FETCHES limited_fetches4 on node3 run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 0: insert for 2000-01-01: [3, 3, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:261555 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SYSTEM RELOAD CONFIG on instance thread 1: insert for 2005-01-02: [1, 5, 2, 10, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 2: insert for 2007-01-03: [6, 10, 5, 7, 9, 6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:163355448 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Stdout:34050785 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SYSTEM RELOAD CONFIG on instance thread 3: insert for 2009-01-04: [8, 5, 8, 8, 5, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:139035 Executing query SELECT * FROM system.quotas ORDER BY name on instance thread 4: insert for 2002-01-05: [8, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance thread 0: insert for 2004-01-01: [2, 8, 4, 5, 9, 5, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance thread 1: insert for 2007-01-02: [3, 9, 9, 10, 9, 8, 8, 1, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SELECT * from test_table on instance thread 2: insert for 2009-01-03: [10, 7, 1, 3, 3, 2, 1, 1, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance thread 13: delete 72 * 9 Executing query ALTER TABLE test_mutations DELETE WHERE x = 9 on node1 thread 3: insert for 2001-01-04: [6, 2, 8, 5, 8, 2, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 4: insert for 2002-01-05: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query DROP QUOTA IF EXISTS qA, qB on instance [gw1] PASSED test_quota/test.py::test_simpliest_quota test_quota/test.py::test_tracking_quota thread 11: delete 28 * 4 Executing query ALTER TABLE test_mutations DELETE WHERE x = 4 on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 0: insert for 2001-01-01: [2, 10, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SYSTEM RELOAD CONFIG on instance thread 1: insert for 2004-01-02: [8, 10, 5, 2, 2, 1, 9, 3, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:264327 thread 2: insert for 2008-01-03: [1, 8, 8, 2, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:174972362 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Stderr: Container roottestpartition-gw5-instance-1 Stopping Stderr: Container roottestpartition-gw5-instance-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_partition/_instances-0-gw5/instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_partition/_instances-0-gw5/instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Stdout:44800529 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SYSTEM RELOAD CONFIG on instance No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 Command:[docker compose --env-file /ClickHouse/tests/integration/test_partition/_instances-0-gw5/.env --project-name roottestpartition-gw5 --file /ClickHouse/tests/integration/test_partition/_instances-0-gw5/instance/docker-compose.yml down --volumes] http://localhost:None "POST /v1.46/exec/f0e2ad23b7ac3cc85e3a6aec490237eb9754e00d693e9064a69bd3002baf7580/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/f0e2ad23b7ac3cc85e3a6aec490237eb9754e00d693e9064a69bd3002baf7580/json HTTP/1.1" 200 586 Stdout:141411 Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vIGxpbWl0cy4gSnVzdCBjYWxjdWxhdGUgcmVzb3VyY2UgdXNhZ2UgZm9yIHRpbWUgaW50ZXJ2YWwuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjA8L3F1ZXJ5X3NlbGVjdHM+CiAgICAgICAgICAgICAgICA8cXVlcnlfaW5zZXJ0cz4wPC9xdWVyeV9pbnNlcnRzPgogICAgICAgICAgICAgICAgPGVycm9ycz4wPC9lcnJvcnM+CiAgICAgICAgICAgICAgICA8cmVhZF9yb3dzPjA8L3JlYWRfcm93cz4KICAgICAgICAgICAgICAgIDxyZXN1bHRfcm93cz4wPC9yZXN1bHRfcm93cz4KICAgICAgICAgICAgPC9pbnRlcnZhbD4KICAgICAgICA8L215UXVvdGE+CiAgICA8L3F1b3Rhcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vIGxpbWl0cy4gSnVzdCBjYWxjdWxhdGUgcmVzb3VyY2UgdXNhZ2UgZm9yIHRpbWUgaW50ZXJ2YWwuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjA8L3F1ZXJ5X3NlbGVjdHM+CiAgICAgICAgICAgICAgICA8cXVlcnlfaW5zZXJ0cz4wPC9xdWVyeV9pbnNlcnRzPgogICAgICAgICAgICAgICAgPGVycm9ycz4wPC9lcnJvcnM+CiAgICAgICAgICAgICAgICA8cmVhZF9yb3dzPjA8L3JlYWRfcm93cz4KICAgICAgICAgICAgICAgIDxyZXN1bHRfcm93cz4wPC9yZXN1bHRfcm93cz4KICAgICAgICAgICAgPC9pbnRlcnZhbD4KICAgICAgICA8L215UXVvdGE+CiAgICA8L3F1b3Rhcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 3: insert for 2007-01-04: [5, 7, 4, 9, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM RELOAD CONFIG on instance thread 0: insert for 2005-01-01: [9, 5, 7, 5, 1, 3, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2008-01-05: [4, 1, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 1: insert for 2000-01-02: [2, 5, 6, 3, 8, 5, 8, 9, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SELECT * FROM system.quotas ORDER BY name on instance thread 2: insert for 2006-01-03: [2, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT * FROM system.quota_limits ORDER BY quota_name, duration on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 3: insert for 2004-01-04: [2, 10, 3, 3, 7, 3, 5, 6, 2, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Stderr: Container roottestpartition-gw5-instance-1 Stopping Stderr: Container roottestpartition-gw5-instance-1 Stopped Stderr: Container roottestpartition-gw5-instance-1 Removing Stderr: Container roottestpartition-gw5-instance-1 Removed Stderr: Network roottestpartition-gw5_default Removing Stderr: Network roottestpartition-gw5_default Removed Cleanup called thread 0: insert for 2008-01-01: [2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 12: delete 59 * 8 Executing query ALTER TABLE test_mutations DELETE WHERE x = 8 on node1 Docker networks for project roottestpartition-gw5 are NETWORK ID NAME DRIVER SCOPE Stdout:265779 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Docker containers for project roottestpartition-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestpartition-gw5 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestpartition-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] Executing query SELECT * from test_table on instance Unstopped containers: {} No running containers for project: roottestpartition-gw5 Trying to prune unused networks... Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Stdout:177930944 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Trying to prune unused images... Command:[docker image prune -f] thread 4: insert for 2010-01-05: [2, 2, 7, 9, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:Total reclaimed space: 0B Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Stdout:47863327 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:7 Command:[docker volume prune -f] thread 1: insert for 2005-01-02: [7, 1, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance Stdout:142929 Executing query SELECT SUM(x) from test_table on instance Stdout:Total reclaimed space: 0B Volumes pruned: 7 Running tests in /ClickHouse/tests/integration/test_reload_zookeeper/test.py test_reload_zookeeper/test.py::test_reload_zookeeper Cluster start called. is_up=False Docker networks for project roottestreloadzookeeper-gw5 are NETWORK ID NAME DRIVER SCOPE Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance Docker containers for project roottestreloadzookeeper-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Executing query SELECT quota_name, quota_key, duration, queries, max_queries, query_selects, max_query_selects, query_inserts, max_query_inserts, errors, max_errors, result_rows, max_result_rows,result_bytes, max_result_bytes, read_rows, max_read_rows, read_bytes, max_read_bytes, max_execution_time, max_failed_sequential_authentications FROM system.quota_usage ORDER BY duration on instance Docker volumes for project roottestreloadzookeeper-gw5 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestreloadzookeeper-gw5 are NETWORK ID NAME DRIVER SCOPE thread 2: insert for 2002-01-03: [4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Docker containers for project roottestreloadzookeeper-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreloadzookeeper-gw5 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreloadzookeeper-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] [gw1] PASSED test_quota/test.py::test_tracking_quota test_quota/test.py::test_users_xml_is_readonly Executing query DROP QUOTA IF EXISTS qA, qB on instance Unstopped containers: {} No running containers for project: roottestreloadzookeeper-gw5 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance Stdout:7 Command:[docker volume prune -f] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVvdGFzPgogICAgICAgIDxteVF1b3RhPgogICAgICAgIDwvbXlRdW90YT4KICAgIDwvcXVvdGFzPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] Stdout:Total reclaimed space: 0B Volumes pruned: 7 Setup directory for instance: node Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files [] to /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/node/configs/config.d Setup database dir /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/node/database Setup logs dir /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/node/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --project-name roottestreloadzookeeper-gw5 --file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml pull] Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Connection dropped: socket connection broken Executing query SYSTEM RELOAD CONFIG on instance thread 3: insert for 2009-01-04: [1, 3, 2, 7, 9, 8, 2, 10, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 0: insert for 2002-01-01: [3, 2, 3, 5, 10, 9, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 1: insert for 2003-01-02: [9, 10, 1, 4, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:3015 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestquota-gw1-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml'] Command:[docker exec roottestquota-gw1-instance-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/users.d/myquota.xml) && echo PGNsaWNraG91c2U+CiAgIDxxdW90YXM+CiAgICAgICAgPG15UXVvdGE+CiAgICAgICAgICAgIDxpbnRlcnZhbD4KICAgICAgICAgICAgICAgIDwhLS0gTGVuZ3RoIG9mIGludGVydmFsID0gMSB5ZWFyIC0tPgogICAgICAgICAgICAgICAgPGR1cmF0aW9uPjMxNTU2OTUyPC9kdXJhdGlvbj4KCiAgICAgICAgICAgICAgICA8IS0tIE5vcm1hbCBsaW1pdHMuIC0tPgogICAgICAgICAgICAgICAgPHF1ZXJpZXM+MTAwMDwvcXVlcmllcz4KICAgICAgICAgICAgICAgIDxxdWVyeV9zZWxlY3RzPjUwMDwvcXVlcnlfc2VsZWN0cz4KICAgICAgICAgICAgICAgIDxxdWVyeV9pbnNlcnRzPjUwMDwvcXVlcnlfaW5zZXJ0cz4KICAgICAgICAgICAgICAgIDxlcnJvcnM+MDwvZXJyb3JzPgogICAgICAgICAgICAgICAgPHJlYWRfcm93cz4xMDAwPC9yZWFkX3Jvd3M+CiAgICAgICAgICAgICAgICA8cmVzdWx0X3Jvd3M+MDwvcmVzdWx0X3Jvd3M+CiAgICAgICAgICAgIDwvaW50ZXJ2YWw+CiAgICAgICAgPC9teVF1b3RhPgogICAgPC9xdW90YXM+CjwvY2xpY2tob3VzZT4K | base64 --decode > /etc/clickhouse-server/users.d/myquota.xml] thread 4: insert for 2009-01-05: [6, 4, 7, 5, 7, 1, 5, 5, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:3015 Executing query select 20 on node1 Executing query SYSTEM RELOAD CONFIG on instance run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 2: insert for 2002-01-03: [10, 9, 9, 9, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:267429 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:181665906 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 0: insert for 2000-01-01: [4, 2, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:51329603 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 3: insert for 2000-01-04: [2, 3, 2, 4, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:144447 thread 1: insert for 2010-01-02: [5, 8, 3, 1, 5, 8, 4, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query DROP QUOTA myQuota on instance Command:[docker compose --env-file /ClickHouse/tests/integration/test_quota/_instances-0-gw1/.env --project-name roottestquota-gw1 --file /ClickHouse/tests/integration/test_quota/_instances-0-gw1/instance/docker-compose.yml stop --timeout 20] [gw1] PASSED test_quota/test.py::test_users_xml_is_readonly Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 Stopping Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 Stopping Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 Stopped Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 Stopped Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Stopping Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Stopping Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Stopping Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Stopped Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Stopped Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] thread 11: delete 83 * 6 Executing query ALTER TABLE test_mutations DELETE WHERE x = 6 on node2 Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/.env --project-name roottestreplicatedmergetreeencryptioncodec-gw7 --file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_encryption_codec/_instances-0-gw7/node2/docker-compose.yml down --volumes] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 4: insert for 2002-01-05: [10, 6, 3, 4, 3, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 0: insert for 2006-01-01: [6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 1: insert for 2010-01-02: [3, 1, 9, 9, 4, 5, 7, 7, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2001-01-03: [2, 6, 10, 2, 3, 4, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 3: insert for 2008-01-04: [7, 10, 3, 1, 2, 1, 8, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:269013 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 4: insert for 2006-01-05: [5, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance Stdout:185478910 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:54954359 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:146163 thread 0: insert for 2002-01-01: [6, 6, 4, 8, 6, 8, 3, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 2: insert for 2004-01-03: [5, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo2 to num2 on node1 thread 3: insert for 2001-01-04: [5, 5, 9, 9, 8, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 Stopping Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 Stopping Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 Stopped Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 Removing Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 Stopped Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 Removing Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node1-1 Removed Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-node2-1 Removed Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Stopping Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Stopping Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Stopping Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Stopped Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Removing Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Stopped Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Removing Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Stopped Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Removing Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo1-1 Removed Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo3-1 Removed Stderr: Container roottestreplicatedmergetreeencryptioncodec-gw7-zoo2-1 Removed Stderr: Network roottestreplicatedmergetreeencryptioncodec-gw7_default Removing Stderr: Network roottestreplicatedmergetreeencryptioncodec-gw7_default Removed Cleanup called Docker networks for project roottestreplicatedmergetreeencryptioncodec-gw7 are NETWORK ID NAME DRIVER SCOPE thread 1: insert for 2010-01-02: [8, 8, 2, 4, 9, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo2 to num2 on node1 Docker containers for project roottestreplicatedmergetreeencryptioncodec-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedmergetreeencryptioncodec-gw7 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmergetreeencryptioncodec-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicatedmergetreeencryptioncodec-gw7 Trying to prune unused networks... thread 13: delete 9 * 6 Executing query ALTER TABLE test_mutations DELETE WHERE x = 6 on node1 Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 4: insert for 2007-01-05: [8, 8, 9, 7, 10, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 0: insert for 2007-01-01: [1, 8, 10, 8, 3, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop Running tests in /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/test.py Cluster start called. is_up=False Docker networks for project roottestreloadmaxtablesizetodrop-gw7 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreloadmaxtablesizetodrop-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreloadmaxtablesizetodrop-gw7 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestreloadmaxtablesizetodrop-gw7 are NETWORK ID NAME DRIVER SCOPE thread 3: insert for 2009-01-04: [9, 5, 8, 3, 2, 3, 8, 7, 8, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Docker containers for project roottestreloadmaxtablesizetodrop-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreloadmaxtablesizetodrop-gw7 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreloadmaxtablesizetodrop-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreloadmaxtablesizetodrop-gw7 Trying to prune unused networks... Executing query DROP TABLE IF EXISTS limited_fetches0 SYNC on node1 Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] thread 2: insert for 2004-01-03: [3, 3, 7, 5, 5, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 Setup directory for instance: node Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_reload_max_table_size_to_drop/configs/max_table_size_to_drop.xml'] to /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/node/configs/config.d Setup database dir /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/node/database Setup logs dir /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/node/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/.env --project-name roottestreloadmaxtablesizetodrop-gw7 --file /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/node/docker-compose.yml pull] thread 1: insert for 2002-01-02: [1, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True thread 0: insert for 2008-01-01: [1, 3, 9, 8, 1, 3, 10, 8, 2] Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 12: delete 30 * 7 Executing query ALTER TABLE test_mutations DELETE WHERE x = 7 on node1 Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper'] Sending request(xid=2): Close() Closing connection to 172.16.2.3:2281 Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.2.2, port:2281, use_ssl:True Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True thread 3: insert for 2009-01-04: [2, 4, 5, 8, 6, 3, 3, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2007-01-05: [9, 2, 3, 1, 9, 4, 1, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 11: delete 99 * 2 Executing query ALTER TABLE test_mutations DELETE WHERE x = 2 on node1 thread 2: insert for 2000-01-03: [5, 7, 10, 3, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance thread 1: insert for 2010-01-02: [9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance thread 0: insert for 2002-01-01: [9, 2, 5, 7, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Executing query DROP TABLE IF EXISTS limited_fetches1 SYNC on node1 thread 3: insert for 2005-01-04: [9, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2010-01-03: [7, 3, 4, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query CREATE TABLE test_add_policy ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 Executing query DROP TABLE IF EXISTS limited_fetches2 SYNC on node1 thread 4: insert for 2003-01-05: [10, 9, 1, 2, 5, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query SYSTEM RELOAD CONFIG on node1 Executing query DROP TABLE IF EXISTS limited_fetches3 SYNC on node1 Executing query SELECT name FROM system.disks on node1 Executing query DROP TABLE IF EXISTS limited_fetches4 SYNC on node1 Executing query SELECT policy_name FROM system.storage_policies on node1 thread 0: insert for 2002-01-01: [3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query DROP TABLE IF EXISTS limited_fetches0 SYNC on node3 thread 1: insert for 2000-01-02: [7, 7, 2, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SELECT volume_name FROM system.storage_policies WHERE policy_name = 'cool_policy' on node1 Executing query SELECT disks FROM system.storage_policies WHERE policy_name = 'cool_policy' on node1 thread 3: insert for 2000-01-04: [2, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2000-01-05: [5, 1, 3, 5, 3, 5, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper'] Sending request(xid=2): Close() Executing query DROP TABLE IF EXISTS test_add_policy on node1 Closing connection to 172.16.2.2:2281 Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.2.4, port:2281, use_ssl:True Connecting to 172.16.2.4(172.16.2.4):2281, use_ssl: True thread 2: insert for 2002-01-03: [5, 1, 5, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 [gw2] PASSED test_reloading_storage_configuration/test.py::test_add_policy test_reloading_storage_configuration/test.py::test_add_volume_to_policy run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 Stdout: PID TTY TIME CMD Stdout: 3015 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] thread 0: insert for 2008-01-01: [1, 8, 6, 10, 9, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 1: insert for 2002-01-02: [8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:3015 Executing query ALTER TABLE test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster RENAME COLUMN foo3 to num2 on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 3: insert for 2000-01-04: [9, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_distributed_parallel_insert_and_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(1000) on node1 thread 2: insert for 2003-01-03: [6, 1, 9, 9, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 4: insert for 2007-01-05: [8, 9, 6, 5, 4, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper'] Sending request(xid=2): Close() thread 0: insert for 2010-01-01: [9, 9, 2, 3, 7, 10, 9, 7, 1, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Closing connection to 172.16.2.4:2281 Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --project-name roottestreloadclientcertificate-gw4 --file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml --file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node2/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --project-name roottestreloadclientcertificate-gw4 --file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml --file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node2/docker-compose.yml up -d --no-recreate] thread 13: delete 59 * 5 Executing query ALTER TABLE test_mutations DELETE WHERE x = 5 on node1 thread 1: insert for 2006-01-02: [9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2001-01-04: [7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2000-01-03: [4, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance thread 4: insert for 2001-01-05: [7, 1, 6, 5, 4, 2, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query DROP TABLE IF EXISTS limited_fetches1 SYNC on node3 thread 0: insert for 2000-01-01: [1, 6, 2, 2, 2, 3, 10, 10, 3, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query DROP TABLE IF EXISTS limited_fetches2 SYNC on node3 Executing query DROP TABLE IF EXISTS limited_fetches3 SYNC on node3 thread 1: insert for 2010-01-02: [9, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node1 thread 2: insert for 2005-01-03: [6, 8, 7, 8, 8, 9, 6, 1, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2009-01-04: [5, 8, 7, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query DROP TABLE IF EXISTS limited_fetches4 SYNC on node3 Executing query CREATE TABLE limited_send_table(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetch_table', '0') ORDER BY tuple() PARTITION BY key SETTINGS max_replicated_sends_network_bandwidth=5242880 on node1 [gw6] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table thread 4: insert for 2006-01-05: [5, 6, 2, 7, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node2 thread 11: delete 22 * 4 Executing query ALTER TABLE test_mutations DELETE WHERE x = 4 on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 0: insert for 2010-01-01: [6, 6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query CREATE TABLE limited_send_table(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_fetch_table', '1') ORDER BY tuple() PARTITION BY key SETTINGS max_replicated_sends_network_bandwidth=5242880 on node2 No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Running Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Running Stderr: Container roottestreloadclientcertificate-gw4-node2-1 Creating Stderr: Container roottestreloadclientcertificate-gw4-node1-1 Creating Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Running Stderr: Container roottestreloadclientcertificate-gw4-node2-1 Created Stderr: Container roottestreloadclientcertificate-gw4-node1-1 Created Stderr: Container roottestreloadclientcertificate-gw4-node2-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-node1-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-node1-1 Started Stderr: Container roottestreloadclientcertificate-gw4-node2-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.2.5... http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/2881f8bcf330e21fc7792c743a750b1785f9e5b10586c3b919075317fdc31bb1/json HTTP/1.1" 200 None http://localhost:None "POST /v1.46/exec/d3311db89f4bc8eb87790a8a9f3fdcffe89aafe526907a0862e7be1be45f5bf7/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/d3311db89f4bc8eb87790a8a9f3fdcffe89aafe526907a0862e7be1be45f5bf7/json HTTP/1.1" 200 586 Executing query SYSTEM STOP FETCHES limited_send_table on node2 thread 1: insert for 2002-01-02: [1, 1, 9, 8, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2003-01-04: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO limited_send_table SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 http://localhost:None "GET /v1.46/containers/2881f8bcf330e21fc7792c743a750b1785f9e5b10586c3b919075317fdc31bb1/json HTTP/1.1" 200 None Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node3 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 2: insert for 2008-01-03: [5, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 http://localhost:None "GET /v1.46/containers/2881f8bcf330e21fc7792c743a750b1785f9e5b10586c3b919075317fdc31bb1/json HTTP/1.1" 200 None thread 0: insert for 2004-01-01: [5, 5, 1, 5, 3, 8, 5, 10, 5, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2005-01-05: [5, 2, 6, 9, 5, 3, 3, 10, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 http://localhost:None "GET /v1.46/containers/2881f8bcf330e21fc7792c743a750b1785f9e5b10586c3b919075317fdc31bb1/json HTTP/1.1" 200 None Executing query INSERT INTO limited_send_table SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 12: delete 43 * 9 Executing query ALTER TABLE test_mutations DELETE WHERE x = 9 on node2 Executing query SELECT count() FROM test_rename_distributed_parallel_insert_and_select WHERE num2 % 1000 > 0 on node4 http://localhost:None "GET /v1.46/containers/2881f8bcf330e21fc7792c743a750b1785f9e5b10586c3b919075317fdc31bb1/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.2.6... http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/3b6a0a9fdb60e752a8933394954dd7f02ca2d11d8d3e1207c79015c1804ee7e9/json HTTP/1.1" 200 None ClickHouse node2 started run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF\n\n \n \n /etc/clickhouse-server/config.d/second_client.crt\n /etc/clickhouse-server/config.d/second_client.key\n true\n true\n sslv2,sslv3\n true\n none\n \n RejectCertificateHandler\n \n \n \n\nEOF'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF /etc/clickhouse-server/config.d/second_client.crt /etc/clickhouse-server/config.d/second_client.key true true sslv2,sslv3 true none RejectCertificateHandler EOF] Executing query INSERT INTO limited_send_table SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 thread 1: insert for 2001-01-02: [7, 6, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 3: insert for 2009-01-04: [4, 5, 7, 7, 6, 2, 10, 1, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'touch /etc/clickhouse-server/config.d/ssl_conf.xml'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c touch /etc/clickhouse-server/config.d/ssl_conf.xml] thread 2: insert for 2005-01-03: [4, 7, 8, 4, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF\n\n \n \n /etc/clickhouse-server/config.d/second_client.crt\n /etc/clickhouse-server/config.d/second_client.key\n true\n true\n sslv2,sslv3\n true\n none\n \n RejectCertificateHandler\n \n \n \n\nEOF'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF /etc/clickhouse-server/config.d/second_client.crt /etc/clickhouse-server/config.d/second_client.key true true sslv2,sslv3 true none RejectCertificateHandler EOF] Executing query INSERT INTO limited_send_table SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query DROP TABLE IF EXISTS test_rename_distributed_parallel_insert_and_select ON CLUSTER test_cluster SYNC on node1 run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'touch /etc/clickhouse-server/config.d/ssl_conf.xml'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c touch /etc/clickhouse-server/config.d/ssl_conf.xml] thread 0: insert for 2009-01-01: [7, 6, 8, 9, 4, 2, 9, 5, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Stopping zookeeper node: zoo1 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo1] Executing query INSERT INTO limited_send_table SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node1 Executing query DROP TABLE IF EXISTS test_rename_distributed_parallel_insert_and_select_replicated ON CLUSTER test_cluster SYNC on node1 thread 4: insert for 2010-01-05: [9, 3, 8, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 1: insert for 2005-01-02: [1, 10, 3, 8, 1, 10, 1, 10, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk \'{print $1 " " $2}\' /proc/net/route | grep 00000000 | awk \'{print $1}\''] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk '{print $1 " " $2}' /proc/net/route | grep 00000000 | awk '{print $1}'] thread 2: insert for 2008-01-03: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:eth0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c grep "^ *eth0:" /proc/net/dev] Stdout: eth0: 321864 2794 0 0 0 0 0 0 193485774 12050 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c grep "^ *eth0:" /proc/net/dev] thread 0: insert for 2001-01-01: [8, 7, 3, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 3: insert for 2002-01-04: [6, 10, 7, 4, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout: eth0: 321864 2794 0 0 0 0 0 0 193485774 12050 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:321864 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 13: delete 47 * 10 Executing query ALTER TABLE test_mutations DELETE WHERE x = 10 on node2 thread 4: insert for 2002-01-05: [5, 5, 9, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:193485774 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk \'{print $1 " " $2}\' /proc/net/route | grep 00000000 | awk \'{print $1}\''] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk '{print $1 " " $2}' /proc/net/route | grep 00000000 | awk '{print $1}'] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 1: insert for 2010-01-02: [2, 7, 3, 1, 9, 5, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:eth0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c grep "^ *eth0:" /proc/net/dev] Stdout:3776 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout: eth0: 130398848 7073 0 0 0 0 0 0 112721 1145 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c grep "^ *eth0:" /proc/net/dev] Stdout:3776 Executing query select 20 on node1 Stdout: eth0: 130398848 7073 0 0 0 0 0 0 112721 1145 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:130398848 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 2: insert for 2009-01-03: [2, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:112721 Executing query SYSTEM START FETCHES limited_send_table on node2 thread 0: insert for 2002-01-01: [6, 4, 1, 3, 10, 10, 1, 10, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 3: insert for 2004-01-04: [9, 1, 7, 7, 8, 6, 8, 5, 3, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:336348 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 1: insert for 2010-01-02: [3, 6, 4, 3, 3, 2, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:198750339 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopped Stopping zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo2] Stdout:135668940 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 4: insert for 2000-01-05: [8, 1, 8, 1, 1, 9, 2, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:131431 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Stderr: Container roottestquota-gw1-instance-1 Stopping Stderr: Container roottestquota-gw1-instance-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_quota/_instances-0-gw1/instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_quota/_instances-0-gw1/instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_quota/_instances-0-gw1/.env --project-name roottestquota-gw1 --file /ClickHouse/tests/integration/test_quota/_instances-0-gw1/instance/docker-compose.yml down --volumes] thread 2: insert for 2001-01-03: [6, 4, 1, 7, 7, 6, 7, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2000-01-01: [10, 4, 10, 10, 9, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2009-01-04: [5, 10, 8, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 1: insert for 2006-01-02: [4, 6, 9, 1, 2, 8, 2, 9, 10, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 4: insert for 2001-01-05: [5, 9, 5, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 11: delete 46 * 8 Executing query ALTER TABLE test_mutations DELETE WHERE x = 8 on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:337008 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:198751288 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 1: insert for 2010-01-02: [5, 6, 10, 6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:136720081 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 2: insert for 2008-01-03: [4, 4, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopped Stopping zookeeper node: zoo3 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo3] Stdout:137041 thread 0: insert for 2005-01-01: [5, 9, 5, 6, 9, 8, 6, 4, 9, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 4: insert for 2005-01-05: [9, 10, 8, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stderr: Container roottestquota-gw1-instance-1 Stopping Stderr: Container roottestquota-gw1-instance-1 Stopped Stderr: Container roottestquota-gw1-instance-1 Removing Stderr: Container roottestquota-gw1-instance-1 Removed Stderr: Network roottestquota-gw1_default Removing Stderr: Network roottestquota-gw1_default Removed Cleanup called thread 3: insert for 2003-01-04: [9, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 [gw3] PASSED test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select test_rename_column/test.py::test_rename_parallel Executing query SELECT value FROM system.build_options WHERE name = 'CXX_FLAGS' on node1 Docker networks for project roottestquota-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestquota-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestquota-gw1 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestquota-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestquota-gw1 Trying to prune unused networks... Trying to prune unused images... Executing query DROP TABLE IF EXISTS test_rename_parallel SYNC on node1 Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper Running tests in /ClickHouse/tests/integration/test_remove_stale_moving_parts/test.py Cluster start called. is_up=False Docker networks for project roottestremovestalemovingparts-gw1 are NETWORK ID NAME DRIVER SCOPE Executing query DROP TABLE IF EXISTS test_rename_parallel SYNC on node2 Docker containers for project roottestremovestalemovingparts-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES thread 12: delete 72 * 1 Executing query ALTER TABLE test_mutations DELETE WHERE x = 1 on node1 Docker volumes for project roottestremovestalemovingparts-gw1 are DRIVER VOLUME NAME Cleanup called Executing query DROP TABLE IF EXISTS test_rename_parallel SYNC on node3 Docker networks for project roottestremovestalemovingparts-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestremovestalemovingparts-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Executing query DROP TABLE IF EXISTS test_rename_parallel SYNC on node4 Docker volumes for project roottestremovestalemovingparts-gw1 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestremovestalemovingparts-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestremovestalemovingparts-gw1 Trying to prune unused networks... thread 1: insert for 2001-01-02: [1, 10, 2, 9, 7, 3, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Trying to prune unused images... Command:[docker image prune -f] thread 2: insert for 2001-01-03: [3, 5, 3, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Executing query CREATE TABLE test_rename_parallel ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_parallel', 'node1') ORDER BY num PARTITION BY num % 100 on node1 Stdout:7 Command:[docker volume prune -f] thread 4: insert for 2001-01-05: [1, 1, 10, 4, 6, 8, 6, 7, 8, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:Total reclaimed space: 0B Volumes pruned: 7 Setup directory for instance: ch1 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_remove_stale_moving_parts/config.xml'] to /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/ch1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/ch1/database Setup logs dir /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/ch1/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper3/coordination', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/minio/certs/public.crt', 'RESOLVER_LOGS': '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/resolver', 'RESOLVER_LOGS_FS': 'bind'} stored in /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --project-name roottestremovestalemovingparts-gw1 --file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/ch1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml pull] Executing query CREATE TABLE test_rename_parallel ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_parallel', 'node2') ORDER BY num PARTITION BY num % 100 on node2 Executing query CREATE TABLE test_rename_parallel ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_parallel', 'node3') ORDER BY num PARTITION BY num % 100 on node3 thread 0: insert for 2009-01-01: [2, 7, 4, 1, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 3: insert for 2008-01-04: [10, 6, 4, 7, 6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:345192 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:204003128 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 4: insert for 2000-01-05: [6, 4, 1, 10, 10, 2, 1, 2, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:140921223 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopped Starting zookeeper node: zoo1 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo1] Executing query CREATE TABLE test_rename_parallel ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_parallel', 'node4') ORDER BY num PARTITION BY num % 100 on node4 thread 1: insert for 2005-01-02: [8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:139945 thread 2: insert for 2000-01-03: [2, 9, 6, 10, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_parallel (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(1000) on node1 thread 13: delete 88 * 3 Executing query ALTER TABLE test_mutations DELETE WHERE x = 3 on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 0: insert for 2010-01-01: [1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2010-01-05: [1, 5, 8, 6, 9, 2, 4, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2008-01-03: [1, 7, 4, 8, 10, 8, 7, 1, 4, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Started Starting zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo2] thread 3: insert for 2008-01-04: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 1: insert for 2004-01-02: [2, 10, 9, 9, 1, 3, 1, 8, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query CREATE TABLE test_add_volume_to_policy ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 Executing query SYSTEM RELOAD CONFIG on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Stdout:351086 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SELECT volume_name FROM system.storage_policies WHERE policy_name = 'cool_policy' on node1 Stdout:209254452 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 0: insert for 2003-01-01: [7, 9, 6, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 2: insert for 2010-01-03: [9, 4, 2, 3, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SELECT disks FROM system.storage_policies WHERE policy_name = 'cool_policy' on node1 Stdout:146172489 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Started Starting zookeeper node: zoo3 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo3] Stdout:147547 thread 3: insert for 2009-01-04: [8, 6, 9, 1, 4, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query DROP TABLE IF EXISTS test_add_volume_to_policy on node1 thread 4: insert for 2006-01-05: [8, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] [gw2] PASSED test_reloading_storage_configuration/test.py::test_add_volume_to_policy test_reloading_storage_configuration/test.py::test_new_policy_works Stdout: PID TTY TIME CMD Stdout: 3776 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] thread 1: insert for 2000-01-02: [1, 8, 5, 9, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Stdout:3776 thread 2: insert for 2009-01-03: [4, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Started get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.2.2, port:2281, use_ssl:True Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused thread 0: insert for 2006-01-01: [3, 10, 2, 10, 8, 2, 9, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance thread 3: insert for 2007-01-04: [5, 3, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query ALTER TABLE test_rename_parallel RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo3 to num2 on node3 Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo2 to foo3 on node2 thread 4: insert for 2009-01-05: [8, 7, 8, 9, 1, 2, 9, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo2 to foo3 on node2 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 1: insert for 2007-01-02: [5, 7, 8, 5, 4, 4, 2, 8, 9, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:355640 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo3 to num2 on node3 Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo3 to num2 on node3 Stdout:213455660 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 12: delete 33 * 9 Executing query ALTER TABLE test_mutations DELETE WHERE x = 9 on node2 Stdout:150373697 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True thread 2: insert for 2001-01-03: [8, 1, 6, 5, 5, 6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo3 to num2 on node3 Stdout:152761 Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo3 to num2 on node3 thread 0: insert for 2002-01-01: [7, 3, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2010-01-04: [2, 1, 4, 4, 3, 7, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 4: insert for 2007-01-05: [6, 3, 3, 9, 8, 3, 7, 2, 9, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 1: insert for 2004-01-02: [2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 2: insert for 2006-01-03: [7, 4, 8, 7, 7, 6, 6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 0: insert for 2005-01-01: [8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 4: insert for 2003-01-05: [5, 10, 1, 7, 7, 3, 10, 10, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 3: insert for 2002-01-04: [1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:359798 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 2: insert for 2010-01-03: [7, 4, 10, 2, 5, 10, 7, 10, 3, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:214506358 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/36747eb286aeb8bfd6d044217471e7a7339a8dedd768be0f7440e4489afb7d21/start HTTP/1.1" 200 0 thread 1: insert for 2007-01-02: [2, 1, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 http://localhost:None "GET /v1.46/exec/36747eb286aeb8bfd6d044217471e7a7339a8dedd768be0f7440e4489afb7d21/json HTTP/1.1" 200 586 Stdout:151424395 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 0: insert for 2008-01-01: [1, 6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:154609 thread 11: delete 30 * 10 Executing query ALTER TABLE test_mutations DELETE WHERE x = 10 on node2 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Connection dropped: socket connection broken thread 4: insert for 2000-01-05: [7, 8, 4, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance thread 3: insert for 2001-01-04: [6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 1: insert for 2000-01-02: [4, 4, 5, 4, 6, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query ALTER TABLE test_rename_parallel RENAME COLUMN num2 to foo2 on node1 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query ALTER TABLE test_rename_parallel RENAME COLUMN num2 to foo2 on node1 thread 2: insert for 2006-01-03: [3, 8, 6, 9, 8, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance Executing query ALTER TABLE test_rename_parallel RENAME COLUMN num2 to foo2 on node1 thread 0: insert for 2002-01-01: [9, 1, 6, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query ALTER TABLE test_rename_parallel RENAME COLUMN num2 to foo2 on node1 Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Executing query SELECT * FROM `test_database`.`postgresql_replica_5` LIMIT 1 FORMAT Null on instance Stdout:366200 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Stdout:219757802 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_5' on instance Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Closing connection to 172.16.2.2:2281 Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.2.3, port:2281, use_ssl:True Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True thread 1: insert for 2006-01-02: [10, 10, 3, 1, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:156675839 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 3: insert for 2005-01-04: [3, 9, 3, 2, 6, 2, 1, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query select * from `postgres_database`.`postgresql_replica_5` order by key; on instance Stdout:161011 thread 2: insert for 2006-01-03: [7, 9, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2007-01-05: [5, 2, 10, 1, 1, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 0: insert for 2007-01-01: [5, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query select * from `test_database`.`postgresql_replica_5` order by key; on instance thread 13: delete 52 * 6 Executing query ALTER TABLE test_mutations DELETE WHERE x = 6 on node1 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Closing connection to 172.16.2.3:2281 Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.2.4, port:2281, use_ssl:True Connecting to 172.16.2.4(172.16.2.4):2281, use_ssl: True thread 1: insert for 2009-01-02: [1, 9, 1, 5, 5, 9, 3, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:4537 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 4: insert for 2009-01-05: [9, 8, 9, 8, 1, 6, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:4537 Executing query select 20 on node1 thread 11: delete 35 * 8 Executing query ALTER TABLE test_mutations DELETE WHERE x = 8 on node2 thread 2: insert for 2000-01-03: [2, 10, 2, 4, 10, 10, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2004-01-04: [2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 12: delete 50 * 5 Executing query ALTER TABLE test_mutations DELETE WHERE x = 5 on node2 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 1: insert for 2010-01-02: [1, 8, 1, 5, 3, 9, 9, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2009-01-01: [8, 6, 7, 1, 7, 9, 4, 7, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Stdout:370886 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:223958680 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo2 to foo3 on node2 Stderr: node Pulling Stderr: node Pulled ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/.env --project-name roottestreloadmaxtablesizetodrop-gw7 --file /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/node/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/.env --project-name roottestreloadmaxtablesizetodrop-gw7 --file /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/node/docker-compose.yml up -d --no-recreate] Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo2 to foo3 on node2 Stdout:160876717 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:165697 Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo2 to foo3 on node2 thread 4: insert for 2002-01-05: [9, 9, 7, 9, 4, 10, 2, 10, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo3 to num2 on node1 Stderr: zoo3 Skipped - Image is already being pulled by node Stderr: zoo1 Skipped - Image is already being pulled by node Stderr: zoo2 Skipped - Image is already being pulled by node Stderr: node Pulling Stderr: node Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper1/log', '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper1/config', '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper1/coordination', '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper2/log', '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper2/config', '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper2/coordination', '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper3/log', '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper3/config', '/ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/keeper3/coordination'] Command:[docker compose --project-name roottestreloadzookeeper-gw5 --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] thread 2: insert for 2001-01-03: [7, 2, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2000-01-01: [4, 1, 3, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 3: insert for 2007-01-04: [6, 1, 8, 4, 10, 4, 9, 9, 10, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) thread 1: insert for 2006-01-02: [1, 7, 8, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Closing connection to 172.16.2.4:2281 Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ['zoo1', 'zoo2', 'zoo3'] run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo -n > /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c echo -n > /var/log/clickhouse-server/clickhouse-server.log] thread 4: insert for 2000-01-05: [5, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo -n > /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c echo -n > /var/log/clickhouse-server/clickhouse-server.log] run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF\n\n \n \n /etc/clickhouse-server/config.d/third_client.crt\n /etc/clickhouse-server/config.d/third_client.key\n true\n true\n sslv2,sslv3\n true\n none\n \n RejectCertificateHandler\n \n \n \n\nEOF'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF /etc/clickhouse-server/config.d/third_client.crt /etc/clickhouse-server/config.d/third_client.key true true sslv2,sslv3 true none RejectCertificateHandler EOF] thread 2: insert for 2005-01-03: [5, 2, 3, 6, 2, 7, 7, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 1: insert for 2002-01-02: [8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 3: insert for 2004-01-04: [7, 7, 2, 6, 9, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query select * from `test_database`.`postgresql_replica_5` order by key; on instance run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'touch /etc/clickhouse-server/config.d/ssl_conf.xml'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c touch /etc/clickhouse-server/config.d/ssl_conf.xml] thread 0: insert for 2003-01-01: [4, 2, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:372140 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF\n\n \n \n /etc/clickhouse-server/config.d/third_client.crt\n /etc/clickhouse-server/config.d/third_client.key\n true\n true\n sslv2,sslv3\n true\n none\n \n RejectCertificateHandler\n \n \n \n\nEOF'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF /etc/clickhouse-server/config.d/third_client.crt /etc/clickhouse-server/config.d/third_client.key true true sslv2,sslv3 true none RejectCertificateHandler EOF] Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_abrupt_connection_loss_while_heavy_replication thread 4: insert for 2002-01-05: [3, 6, 8, 3, 9, 8, 9, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:226059482 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'touch /etc/clickhouse-server/config.d/ssl_conf.xml'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c touch /etc/clickhouse-server/config.d/ssl_conf.xml] Stderr: zoo1 Skipped - Image is already being pulled by ch1 Stderr: zoo2 Skipped - Image is already being pulled by ch1 Stderr: zoo3 Skipped - Image is already being pulled by ch1 Stderr: proxy2 Skipped - Image is already being pulled by proxy1 Stderr: resolver Pulling Stderr: minio1 Pulling Stderr: proxy1 Pulling Stderr: ch1 Pulling Stderr: resolver Pulled Stderr: minio1 Pulled Stderr: proxy1 Pulled Stderr: ch1 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper1/log', '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper1/config', '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper1/coordination', '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper2/log', '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper2/config', '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper2/coordination', '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper3/log', '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper3/config', '/ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/keeper3/coordination'] Command:[docker compose --project-name roottestremovestalemovingparts-gw1 --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stdout:166128095 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:171571 thread 1: insert for 2010-01-02: [4, 9, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2004-01-03: [2, 4, 8, 4, 3, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance thread 0: insert for 2008-01-01: [1, 3, 5, 6, 3, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2009-01-04: [5, 4, 7, 10, 3, 10, 5, 6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 4: insert for 2008-01-05: [7, 2, 6, 5, 9, 5, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stderr: Network roottestreloadmaxtablesizetodrop-gw7_default Creating Stderr: Network roottestreloadmaxtablesizetodrop-gw7_default Created Stderr: Container roottestreloadmaxtablesizetodrop-gw7-node-1 Creating Stderr: Container roottestreloadmaxtablesizetodrop-gw7-node-1 Created Stderr: Container roottestreloadmaxtablesizetodrop-gw7-node-1 Starting Stderr: Container roottestreloadmaxtablesizetodrop-gw7-node-1 Started ClickHouse instance created get_instance_ip instance_name=node http://localhost:None "GET /v1.46/containers/roottestreloadmaxtablesizetodrop-gw7-node-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node thread 1: insert for 2006-01-02: [3, 2, 5, 7, 9, 10, 8, 1, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 http://localhost:None "GET /v1.46/containers/roottestreloadmaxtablesizetodrop-gw7-node-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node, ip: 172.16.3.2... http://localhost:None "GET /v1.46/containers/roottestreloadmaxtablesizetodrop-gw7-node-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/6759f8e310bd054d173f459b68466067456dcf442eaee8c00a4ad06d78ee6afd/json HTTP/1.1" 200 None Stderr:time="2025-04-02T03:32:10Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreloadzookeeper-gw5_default Creating Stderr: Network roottestreloadzookeeper-gw5_default Created Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Creating Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Creating Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Creating Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Created Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Created Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Created Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Starting Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Starting Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Starting Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Started Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Started Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Started Stderr:time="2025-04-02T03:32:11Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:32:11Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.5.3, port:2181, use_ssl:False Connecting to 172.16.5.3(172.16.5.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 2: insert for 2002-01-03: [4, 6, 4, 5, 3, 5, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 11: delete 61 * 2 Executing query ALTER TABLE test_mutations DELETE WHERE x = 2 on node1 Stdout:378080 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] http://localhost:None "GET /v1.46/containers/6759f8e310bd054d173f459b68466067456dcf442eaee8c00a4ad06d78ee6afd/json HTTP/1.1" 200 None Connecting to 172.16.5.3(172.16.5.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Stdout:230260558 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:167178595 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] http://localhost:None "GET /v1.46/containers/6759f8e310bd054d173f459b68466067456dcf442eaee8c00a4ad06d78ee6afd/json HTTP/1.1" 200 None Executing query ALTER TABLE test_rename_parallel RENAME COLUMN foo2 to num2 on node1 Stdout:172891 Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance thread 13: delete 31 * 9 Executing query ALTER TABLE test_mutations DELETE WHERE x = 9 on node2 Connecting to 172.16.5.3(172.16.5.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 4: insert for 2002-01-05: [10, 4, 7, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node2 http://localhost:None "GET /v1.46/containers/6759f8e310bd054d173f459b68466067456dcf442eaee8c00a4ad06d78ee6afd/json HTTP/1.1" 200 None thread 1: insert for 2001-01-02: [4, 10, 5, 1, 3, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 0: insert for 2007-01-01: [8, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 3: insert for 2009-01-04: [2] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 2: insert for 2004-01-03: [1, 5, 4, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 http://localhost:None "GET /v1.46/containers/6759f8e310bd054d173f459b68466067456dcf442eaee8c00a4ad06d78ee6afd/json HTTP/1.1" 200 None test_postgresql_replica_database_engine_1/test.py::test_abrupt_server_restart_while_heavy_replication Stderr:time="2025-04-02T03:32:11Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestremovestalemovingparts-gw1_default Creating Stderr: Network roottestremovestalemovingparts-gw1_default Created Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Creating Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Creating Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Creating Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Created Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Created Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Created Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Started Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Started Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Started Stderr:time="2025-04-02T03:32:12Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:32:12Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.7.3, port:2181, use_ssl:False Executing query SELECT count() FROM test_rename_parallel WHERE num2 % 1000 > 0 on node1 Connecting to 172.16.7.3(172.16.7.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query CREATE TABLE test_new_policy_works ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query DROP TABLE IF EXISTS test_rename_parallel SYNC on node1 Executing query SYSTEM RELOAD CONFIG on node1 http://localhost:None "GET /v1.46/containers/6759f8e310bd054d173f459b68466067456dcf442eaee8c00a4ad06d78ee6afd/json HTTP/1.1" 200 None Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance ClickHouse node started Executing query CREATE TABLE test(date Date, id UInt32) ENGINE = MergeTree() PARTITION BY date ORDER BY id on node Connecting to 172.16.7.3(172.16.7.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query ALTER TABLE test_new_policy_works MODIFY SETTING storage_policy='cool_policy' on node1 Executing query SHOW DATABASES on instance Executing query INSERT INTO test VALUES (now(), 0) on node Connecting to 172.16.5.3(172.16.5.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SYSTEM RELOAD CONFIG on node1 Executing query DROP TABLE IF EXISTS test_rename_parallel SYNC on node2 Executing query ALTER TABLE test_new_policy_works MODIFY SETTING storage_policy='cool_policy' on node1 Executing query DROP TABLE IF EXISTS limited_send_table SYNC on node1 thread 12: delete 57 * 4 Executing query ALTER TABLE test_mutations DELETE WHERE x = 4 on node2 thread 1: insert for 2005-01-02: [1, 6, 7, 1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 0: insert for 2009-01-01: [1, 3, 1, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 2: insert for 2005-01-03: [2, 7, 1, 2, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO TABLE test_new_policy_works VALUES (1) on node1 Executing query DROP TABLE IF EXISTS test_rename_parallel SYNC on node3 thread 4: insert for 2010-01-05: [6, 7, 3, 5, 6, 9, 6, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 3: insert for 2007-01-04: [10] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Connecting to 172.16.7.3(172.16.7.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT disk_name FROM system.parts WHERE active = 1 AND table = 'test_new_policy_works' on node1 Executing query DROP TABLE IF EXISTS test_rename_parallel SYNC on node4 Executing query DROP TABLE IF EXISTS test_new_policy_works on node1 Executing query DROP TABLE IF EXISTS limited_send_table SYNC on node2 [gw2] PASSED test_reloading_storage_configuration/test.py::test_new_policy_works test_reloading_storage_configuration/test.py::test_remove_disk run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] Stdout: PID TTY TIME CMD Stdout: 4537 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 1: insert for 2001-01-02: [5, 7, 7, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 [gw3] PASSED test_rename_column/test.py::test_rename_parallel test_rename_column/test.py::test_rename_parallel_same_node Executing query SELECT value FROM system.build_options WHERE name = 'CXX_FLAGS' on node1 [gw6] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table Executing query CREATE TABLE limited_sends0(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_sends0', '0') ORDER BY tuple() PARTITION BY key on node1 test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server Connecting to 172.16.5.3(172.16.5.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 3: insert for 2007-01-04: [6, 10, 3, 10, 7, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Stdout:4537 Executing query DROP TABLE IF EXISTS test_rename_parallel_same_node SYNC on node1 thread 0: insert for 2003-01-01: [2, 4, 4, 8, 1, 5, 8, 7, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query CREATE TABLE limited_sends1(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_sends1', '0') ORDER BY tuple() PARTITION BY key on node1 Executing query DROP TABLE IF EXISTS test_rename_parallel_same_node SYNC on node2 thread 2: insert for 2002-01-03: [10, 1, 4, 6, 10, 3, 5, 1, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query CREATE TABLE limited_sends2(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_sends2', '0') ORDER BY tuple() PARTITION BY key on node1 thread 4: insert for 2001-01-05: [3, 4, 9, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query DROP TABLE IF EXISTS test_rename_parallel_same_node SYNC on node3 Executing query CREATE TABLE limited_sends3(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_sends3', '0') ORDER BY tuple() PARTITION BY key on node1 Executing query DROP TABLE IF EXISTS test_rename_parallel_same_node SYNC on node4 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query CREATE TABLE limited_sends4(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_sends4', '0') ORDER BY tuple() PARTITION BY key on node1 Executing query CREATE TABLE test_rename_parallel_same_node ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_parallel_same_node', 'node1') ORDER BY num PARTITION BY num % 100 on node1 Executing query CREATE TABLE limited_sends0(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_sends0', '1') ORDER BY tuple() PARTITION BY key on node3 Executing query CREATE TABLE test_rename_parallel_same_node ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_parallel_same_node', 'node2') ORDER BY num PARTITION BY num % 100 on node2 thread 1: insert for 2000-01-02: [5, 1, 3, 1, 9, 1, 5, 10, 1, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query CREATE TABLE limited_sends1(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_sends1', '1') ORDER BY tuple() PARTITION BY key on node3 Executing query CREATE TABLE test_rename_parallel_same_node ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_parallel_same_node', 'node3') ORDER BY num PARTITION BY num % 100 on node3 thread 0: insert for 2005-01-01: [1, 8, 3, 6, 2, 3, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Connecting to 172.16.7.3(172.16.7.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 3: insert for 2006-01-04: [1, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 thread 4: insert for 2010-01-05: [9, 5, 9, 6, 10, 10, 9, 7, 4, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query CREATE TABLE limited_sends2(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_sends2', '1') ORDER BY tuple() PARTITION BY key on node3 Executing query CREATE TABLE test_rename_parallel_same_node ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_parallel_same_node', 'node4') ORDER BY num PARTITION BY num % 100 on node4 thread 13: delete 29 * 10 Executing query ALTER TABLE test_mutations DELETE WHERE x = 10 on node2 Executing query CREATE TABLE limited_sends3(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_sends3', '1') ORDER BY tuple() PARTITION BY key on node3 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_parallel_same_node (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(1000) on node1 thread 2: insert for 2005-01-03: [1, 4, 7, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query CREATE TABLE limited_sends4(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/limited_sends4', '1') ORDER BY tuple() PARTITION BY key on node3 thread 1: insert for 2007-01-02: [5, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query SYSTEM STOP FETCHES limited_sends0 on node1 Executing query INSERT INTO limited_sends0 SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 thread 4: insert for 2009-01-05: [9, 1, 10, 1, 2, 8, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 0: insert for 2004-01-01: [3, 7, 7, 5, 3, 4, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query INSERT INTO limited_sends0 SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 thread 3: insert for 2003-01-04: [1, 1, 1, 4, 6, 1, 7, 1, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node2 thread 2: insert for 2002-01-03: [8, 1, 7, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node1 Executing query INSERT INTO limited_sends0 SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 1: insert for 2006-01-02: [7, 5, 2, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node2 Executing query INSERT INTO limited_sends0 SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/00196c32176b141c45014d9f7527e6510a86d453961cfaf7e8d8f9bf10936552/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/00196c32176b141c45014d9f7527e6510a86d453961cfaf7e8d8f9bf10936552/json HTTP/1.1" 200 586 Executing query INSERT INTO limited_sends0 SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Connecting to 172.16.7.3(172.16.7.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.5.3(172.16.5.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SYSTEM STOP FETCHES limited_sends1 on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query INSERT INTO limited_sends1 SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo2 to foo3 on node1 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo3 to num2 on node1 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo2 to foo3 on node1 Executing query INSERT INTO limited_sends1 SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo3 to num2 on node1 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo3 to num2 on node1 Executing query INSERT INTO limited_sends1 SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo3 to num2 on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo3 to num2 on node1 Executing query INSERT INTO limited_sends1 SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query INSERT INTO limited_sends1 SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node1 Executing query SYSTEM STOP FETCHES limited_sends2 on node1 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node2 Executing query INSERT INTO limited_sends2 SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Total mutations: 43 Executing query SELECT mutation_id, command, parts_to_do, is_done FROM system.mutations WHERE table = 'test_mutations' FORMAT TSVWithNames on node1 mutation_id command parts_to_do is_done 0000000000 (DELETE WHERE x = 3) 0 1 0000000001 (DELETE WHERE x = 7) 0 1 0000000002 (DELETE WHERE x = 9) 0 1 0000000003 (DELETE WHERE x = 4) 0 1 0000000004 (DELETE WHERE x = 8) 0 1 0000000005 (DELETE WHERE x = 6) 0 1 0000000006 (DELETE WHERE x = 1) 0 1 0000000007 (DELETE WHERE x = 9) 0 1 0000000008 (DELETE WHERE x = 5) 0 1 0000000009 (DELETE WHERE x = 2) 0 1 0000000010 (DELETE WHERE x = 8) 0 1 0000000011 (DELETE WHERE x = 10) 0 1 0000000012 (DELETE WHERE x = 1) 0 1 0000000013 (DELETE WHERE x = 7) 0 1 0000000014 (DELETE WHERE x = 4) 0 1 0000000015 (DELETE WHERE x = 3) 0 1 0000000016 (DELETE WHERE x = 5) 0 1 0000000017 (DELETE WHERE x = 7) 0 1 0000000018 (DELETE WHERE x = 1) 0 1 0000000019 (DELETE WHERE x = 10) 0 1 0000000020 (DELETE WHERE x = 9) 0 1 0000000021 (DELETE WHERE x = 4) 0 1 0000000022 (DELETE WHERE x = 8) 0 1 0000000023 (DELETE WHERE x = 6) 0 1 0000000024 (DELETE WHERE x = 6) 0 1 0000000025 (DELETE WHERE x = 7) 0 1 0000000026 (DELETE WHERE x = 2) 0 1 0000000027 (DELETE WHERE x = 5) 0 1 0000000028 (DELETE WHERE x = 4) 0 1 0000000029 (DELETE WHERE x = 9) 0 1 0000000030 (DELETE WHERE x = 10) 0 1 0000000031 (DELETE WHERE x = 8) 0 1 0000000032 (DELETE WHERE x = 1) 0 1 0000000033 (DELETE WHERE x = 3) 0 1 0000000034 (DELETE WHERE x = 9) 0 1 0000000035 (DELETE WHERE x = 10) 0 1 0000000036 (DELETE WHERE x = 6) 0 1 0000000037 (DELETE WHERE x = 8) 0 1 0000000038 (DELETE WHERE x = 5) 0 1 0000000039 (DELETE WHERE x = 2) 0 1 0000000040 (DELETE WHERE x = 9) 0 1 0000000041 (DELETE WHERE x = 4) 0 1 0000000042 (DELETE WHERE x = 10) 0 1 Executing query SELECT mutation_id, command, parts_to_do, is_done FROM system.mutations WHERE table = 'test_mutations' FORMAT TSVWithNames on node2 Executing query INSERT INTO limited_sends2 SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] mutation_id command parts_to_do is_done 0000000000 (DELETE WHERE x = 3) 0 1 0000000001 (DELETE WHERE x = 7) 0 1 0000000002 (DELETE WHERE x = 9) 0 1 0000000003 (DELETE WHERE x = 4) 0 1 0000000004 (DELETE WHERE x = 8) 0 1 0000000005 (DELETE WHERE x = 6) 0 1 0000000006 (DELETE WHERE x = 1) 0 1 0000000007 (DELETE WHERE x = 9) 0 1 0000000008 (DELETE WHERE x = 5) 0 1 0000000009 (DELETE WHERE x = 2) 0 1 0000000010 (DELETE WHERE x = 8) 0 1 0000000011 (DELETE WHERE x = 10) 0 1 0000000012 (DELETE WHERE x = 1) 0 1 0000000013 (DELETE WHERE x = 7) 0 1 0000000014 (DELETE WHERE x = 4) 0 1 0000000015 (DELETE WHERE x = 3) 0 1 0000000016 (DELETE WHERE x = 5) 0 1 0000000017 (DELETE WHERE x = 7) 0 1 0000000018 (DELETE WHERE x = 1) 0 1 0000000019 (DELETE WHERE x = 10) 0 1 0000000020 (DELETE WHERE x = 9) 0 1 0000000021 (DELETE WHERE x = 4) 0 1 0000000022 (DELETE WHERE x = 8) 0 1 0000000023 (DELETE WHERE x = 6) 0 1 0000000024 (DELETE WHERE x = 6) 0 1 0000000025 (DELETE WHERE x = 7) 0 1 0000000026 (DELETE WHERE x = 2) 0 1 0000000027 (DELETE WHERE x = 5) 0 1 0000000028 (DELETE WHERE x = 4) 0 1 0000000029 (DELETE WHERE x = 9) 0 1 0000000030 (DELETE WHERE x = 10) 0 1 0000000031 (DELETE WHERE x = 8) 0 1 0000000032 (DELETE WHERE x = 1) 0 1 0000000033 (DELETE WHERE x = 3) 0 1 0000000034 (DELETE WHERE x = 9) 0 1 0000000035 (DELETE WHERE x = 10) 0 1 0000000036 (DELETE WHERE x = 6) 0 1 0000000037 (DELETE WHERE x = 8) 0 1 0000000038 (DELETE WHERE x = 5) 0 1 0000000039 (DELETE WHERE x = 2) 0 1 0000000040 (DELETE WHERE x = 9) 0 1 0000000041 (DELETE WHERE x = 4) 0 1 0000000042 (DELETE WHERE x = 10) 0 1 Executing query SELECT sum(x) FROM test_mutations on node1 Stdout:5299 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query INSERT INTO limited_sends2 SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Stdout:5299 Executing query select 20 on node1 Executing query SELECT sum(x) FROM test_mutations on node2 [gw9] PASSED test_replicated_mutations/test.py::test_mutations Executing query DROP TABLE IF EXISTS test_mutations SYNC on node1 test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes0] Executing query INSERT INTO limited_sends2 SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN num2 to foo2 on node1 Executing query INSERT INTO limited_sends2 SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN num2 to foo2 on node1 Connecting to 172.16.7.3(172.16.7.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query SYSTEM STOP FETCHES limited_sends3 on node1 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.7.2, port:2181, use_ssl:False Connecting to 172.16.7.2(172.16.7.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Executing query INSERT INTO limited_sends3 SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.7.4, port:2181, use_ssl:False Connecting to 172.16.7.4(172.16.7.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query INSERT INTO limited_sends3 SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query DROP TABLE IF EXISTS test_mutations SYNC on node2 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') Trying to create Minio instance by command docker compose --project-name roottestremovestalemovingparts-gw1 --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --verbose up -d Command:[docker compose --project-name roottestremovestalemovingparts-gw1 --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --verbose up -d] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query INSERT INTO limited_sends3 SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query INSERT INTO limited_sends3 SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query INSERT INTO limited_sends3 SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query SYSTEM STOP FETCHES limited_sends4 on node1 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps -C clickhouse] Executing query INSERT INTO limited_sends4 SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Stdout: PID TTY TIME CMD Stdout: 9 ? 00:00:04 clickhouse run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill clickhouse'] Command:[docker exec -u root roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c pkill clickhouse] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:9 Executing query INSERT INTO limited_sends4 SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo2 to foo3 on node1 Executing query DROP TABLE IF EXISTS test_mutations SYNC on node3 Executing query INSERT INTO limited_sends4 SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo2 to foo3 on node1 Executing query DROP TABLE IF EXISTS test_mutations SYNC on node4 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo2 to foo3 on node1 Executing query INSERT INTO limited_sends4 SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo3 to num2 on node1 Executing query DROP TABLE IF EXISTS test_mutations SYNC on node5 Executing query INSERT INTO limited_sends4 SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(150) on node3 Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node1 Connecting to 172.16.5.3(172.16.5.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk \'{print $1 " " $2}\' /proc/net/route | grep 00000000 | awk \'{print $1}\''] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk '{print $1 " " $2}' /proc/net/route | grep 00000000 | awk '{print $1}'] Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node2 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.5.4, port:2181, use_ssl:False Connecting to 172.16.5.4(172.16.5.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Stdout:eth0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Stderr:time="2025-04-02T03:32:15Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Volume "roottestremovestalemovingparts-gw1_data1-1" Creating Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c grep "^ *eth0:" /proc/net/dev] Stderr: Volume "roottestremovestalemovingparts-gw1_data1-1" Created Stderr:time="2025-04-02T03:32:15Z" level=warning msg="Found orphan containers ([roottestremovestalemovingparts-gw1-zoo2-1 roottestremovestalemovingparts-gw1-zoo3-1 roottestremovestalemovingparts-gw1-zoo1-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up." Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Creating Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Creating Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Created Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Created Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Creating Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Creating Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Created Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Created Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Started Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Started Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Started Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Started Stderr:time="2025-04-02T03:32:17Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:32:17Z" level=debug msg="otel error" error="" Trying to connect to Minio... get_instance_ip instance_name=minio1 http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-minio1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=proxy1 http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-proxy1-1/json HTTP/1.1" 200 None Starting new HTTP connection (1): 172.16.7.8:9001 Incremented Retry for (url='/'): Retry(total=2, connect=None, read=None, redirect=None, status=None) Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / Starting new HTTP connection (2): 172.16.7.8:9001 Incremented Retry for (url='/'): Retry(total=1, connect=None, read=None, redirect=None, status=None) Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / Starting new HTTP connection (3): 172.16.7.8:9001 Incremented Retry for (url='/'): Retry(total=0, connect=None, read=None, redirect=None, status=None) Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')': / Starting new HTTP connection (4): 172.16.7.8:9001 Can't connect to Minio: HTTPConnectionPool(host='172.16.7.8', port=9001): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused')) Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node3 Stdout: eth0: 483294 4303 0 0 0 0 0 0 234566499 13836 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 Stdout: eth0: 483294 4303 0 0 0 0 0 0 234566499 13836 0 0 0 0 0 0 http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-zoo3-1/json HTTP/1.1" 200 None run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] get_kazoo_client: zoo3, ip:172.16.5.2, port:2181, use_ssl:False Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Connecting to 172.16.5.2(172.16.5.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node4 Stdout:483380 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE MergeTree() ORDER BY x PARTITION BY toYYYYMM(d) on node5 Stdout:234566643 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk \'{print $1 " " $2}\' /proc/net/route | grep 00000000 | awk \'{print $1}\''] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk '{print $1 " " $2}' /proc/net/route | grep 00000000 | awk '{print $1}'] Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --project-name roottestreloadzookeeper-gw5 --file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --project-name roottestreloadzookeeper-gw5 --file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml up -d --no-recreate] Executing query CREATE TABLE test_remove_disk ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 Stdout:eth0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT name FROM system.disks on node1 Stdout: eth0: 63082913 5824 0 0 0 0 0 0 448058 2560 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SYSTEM RELOAD CONFIG on node1 Stdout: eth0: 63082913 5824 0 0 0 0 0 0 448058 2560 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:63082913 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query DROP TABLE test on node Stdout:9 Executing query SYSTEM RELOAD CONFIG on node Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:448058 Executing query SYSTEM START FETCHES limited_sends0 on node1 Executing query SELECT name FROM system.disks on node1 Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SYSTEM START FETCHES limited_sends1 on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c cat /var/log/clickhouse-server/clickhouse-server.log] Stdout:2025.04.02 03:32:14.090084 [ 5299 ] {} SentryWriter: Sending crash reports is disabled Stdout:2025.04.02 03:32:14.136646 [ 5299 ] {} Application: Starting ClickHouse 25.2.2.96963.altinityantalya.96963 (revision: 54496, git hash: 25f7c2a766916d92d37aee82d9defa5f14b86ec8, build id: 4783FC52DD13D5DFB0294BDDD711047195FEB5A6), PID 5299 Stdout:2025.04.02 03:32:14.136765 [ 5299 ] {} Application: starting up Stdout:2025.04.02 03:32:14.136773 [ 5299 ] {} Application: OS name: Linux, version: 5.15.0-130-generic, architecture: x86_64 Stdout:2025.04.02 03:32:14.136892 [ 5299 ] {} Jemalloc: Value for background_thread set to true (from true) Stdout:2025.04.02 03:32:14.141110 [ 5299 ] {} Application: Available RAM: 30.60 GiB; logical cores: 16; used cores: 16. Stdout:2025.04.02 03:32:14.141132 [ 5299 ] {} Application: Available CPU instruction sets: SSE, SSE2, SSE3, SSSE3, SSE41, SSE42, F16C, POPCNT, BMI1, BMI2, PCLMUL, AES, AVX, FMA, AVX2, SHA, ADX, RDRAND, RDSEED, RDTSCP, CLFLUSHOPT, CLWB, XSAVE, OSXSAVE Stdout:2025.04.02 03:32:14.141161 [ 5299 ] {} Pipe: Pipe capacity is 1.00 MiB Stdout:2025.04.02 03:32:14.141747 [ 5299 ] {} CgroupsReader: Will create cgroup reader from '/sys/fs/cgroup/' (cgroups version: v2) Stdout:2025.04.02 03:32:14.141867 [ 5299 ] {} AsynchronousMetrics: Scanning /sys/class/thermal Stdout:2025.04.02 03:32:14.141878 [ 5299 ] {} AsynchronousMetrics: Scanning /sys/block Stdout:2025.04.02 03:32:14.141935 [ 5299 ] {} AsynchronousMetrics: Scanning /sys/devices/system/edac Stdout:2025.04.02 03:32:14.141945 [ 5299 ] {} AsynchronousMetrics: Scanning /sys/class/hwmon Stdout:2025.04.02 03:32:14.142003 [ 5299 ] {} StatusFile: Status file /var/lib/clickhouse/status already exists - unclean restart. Contents: Stdout:PID: 4537 Stdout:Started at: 2025-04-02 03:32:09 Stdout:Revision: 54496 Stdout: Stdout:2025.04.02 03:32:14.308523 [ 5299 ] {} Application: Integrity check of the executable successfully passed (checksum: 2D3F861318CFEDFBBB742B17F83CC397) Stdout:2025.04.02 03:32:14.308648 [ 5299 ] {} Application: Will do mlock to prevent executable memory from being paged out. It may take a few seconds. Stdout:2025.04.02 03:32:14.313186 [ 5299 ] {} Application: The memory map of clickhouse executable has been mlock'ed, total 287.17 MiB Stdout:2025.04.02 03:32:14.314266 [ 5299 ] {} MemoryWorker: Starting background memory thread with period of 50ms, using Cgroups as source Stdout:2025.04.02 03:32:14.314351 [ 5299 ] {} BackgroundSchedulePool/BgSchPool: Create BackgroundSchedulePool with 512 threads Stdout:2025.04.02 03:32:14.366456 [ 5315 ] {} MemoryTracker: Correcting the value of global memory tracker from 3.21 MiB to 105.32 MiB Stdout:2025.04.02 03:32:14.372890 [ 5299 ] {} Application: rlimit on number of file descriptors is 1048576 Stdout:2025.04.02 03:32:14.372935 [ 5299 ] {} Application: rlimit on number of threads is 18446744073709551615 Stdout:2025.04.02 03:32:14.372956 [ 5299 ] {} Application: Initializing DateLUT. Stdout:2025.04.02 03:32:14.372963 [ 5299 ] {} Application: Initialized DateLUT with time zone 'Etc/UTC'. Stdout:2025.04.02 03:32:14.372892 [ 5827 ] {} CancellationChecker: Started worker function Stdout:2025.04.02 03:32:14.373006 [ 5299 ] {} Context: Setting up /var/lib/clickhouse/tmp/ to store temporary data in it Stdout:2025.04.02 03:32:14.373277 [ 5299 ] {} Application: Configuration parameter 'interserver_http_host' doesn't exist or exists and empty. Will use 'node1' as replica host. Stdout:2025.04.02 03:32:14.373302 [ 5299 ] {} Application: Initializing interserver credentials. Stdout:2025.04.02 03:32:14.373444 [ 5299 ] {} NamedCollectionsMetadataStorage: Using local storage for named collections at path: /var/lib/clickhouse/named_collections Stdout:2025.04.02 03:32:14.373473 [ 5299 ] {} FileCacheFactory: Will load 0 caches from default cache config Stdout:2025.04.02 03:32:14.375680 [ 5299 ] {} ConfigReloader: Loading config '/etc/clickhouse-server/config.xml' Stdout:2025.04.02 03:32:14.375696 [ 5299 ] {} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/config.xml'. Stdout:2025.04.02 03:32:14.376580 [ 5299 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:14.376691 [ 5299 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:14.376776 [ 5299 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_enable_keeper_async_replication.xml'. Stdout:2025.04.02 03:32:14.376830 [ 5299 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_instance_config.xml'. Stdout:2025.04.02 03:32:14.376977 [ 5299 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/logs_config.xml'. Stdout:2025.04.02 03:32:14.377072 [ 5299 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/storage_configuration.xml'. Stdout:2025.04.02 03:32:14.381472 [ 5299 ] {} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'. Stdout:2025.04.02 03:32:14.381565 [ 5299 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performing update on configuration Stdout:2025.04.02 03:32:14.387899 [ 5299 ] {} Application: Changed setting 'max_server_memory_usage' to 27.54 GiB (30.60 GiB available memory * 0.90 max_server_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:14.387912 [ 5299 ] {} Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.30 GiB (30.60 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:14.387917 [ 5299 ] {} Application: Merges and mutations memory limit is set to 15.30 GiB Stdout:2025.04.02 03:32:14.388045 [ 5299 ] {} Application: Setting max_remote_read_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:14.388051 [ 5299 ] {} Application: Setting max_remote_write_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:14.388058 [ 5299 ] {} Application: ConcurrencyControl limit is set to 32 CPU slots with 'fair_round_robin' scheduler Stdout:2025.04.02 03:32:14.388068 [ 5299 ] {} BackgroundSchedulePool/BgBufSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:14.392326 [ 5299 ] {} BackgroundSchedulePool/BgMBSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:14.397965 [ 5299 ] {} BackgroundSchedulePool/BgDistSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:14.399998 [ 5299 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration Stdout:2025.04.02 03:32:14.400045 [ 5299 ] {} ConfigReloader: Config reload interval set to 2000ms Stdout:2025.04.02 03:32:14.400592 [ 5299 ] {} Application: Listening for replica communication (interserver): http://0.0.0.0:9009 Stdout:2025.04.02 03:32:14.402762 [ 5299 ] {} ConfigReloader: Loading config '/etc/clickhouse-server/users.xml' Stdout:2025.04.02 03:32:14.402800 [ 5299 ] {} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/users.xml'. Stdout:2025.04.02 03:32:14.403176 [ 5299 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:14.403240 [ 5299 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:14.403298 [ 5299 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/users.d/0_common_instance_users.xml'. Stdout:2025.04.02 03:32:14.403339 [ 5299 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/users.d/0_random_settings.xml'. Stdout:2025.04.02 03:32:14.403876 [ 5299 ] {} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'. Stdout:2025.04.02 03:32:14.403897 [ 5299 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performing update on configuration Stdout:2025.04.02 03:32:14.404534 [ 5299 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performed update on configuration Stdout:2025.04.02 03:32:14.404549 [ 5299 ] {} ConfigReloader: Config reload interval set to 2000ms Stdout:2025.04.02 03:32:14.404811 [ 5299 ] {} Access(user directories): Added users_xml access storage 'users_xml', path: /etc/clickhouse-server/users.xml Stdout:2025.04.02 03:32:14.404932 [ 5299 ] {} Access(user directories): Added local_directory access storage 'local_directory', path: /var/lib/clickhouse/access/ Stdout:2025.04.02 03:32:14.405028 [ 5299 ] {} CgroupsMemoryUsageObserver: Started cgroup current memory usage observer thread Stdout:2025.04.02 03:32:14.405366 [ 5882 ] {} CgroupsMemoryUsageObserver: Memory amount initially available to the process is 30.60 GiB Stdout:2025.04.02 03:32:14.408447 [ 5299 ] {} Context: Initialized background executor for merges and mutations with num_threads=16, num_tasks=32, scheduling_policy=round_robin Stdout:2025.04.02 03:32:14.409197 [ 5299 ] {} Context: Initialized background executor for move operations with num_threads=8, num_tasks=8 Stdout:2025.04.02 03:32:14.410472 [ 5299 ] {} Context: Initialized background executor for fetches with num_threads=16, num_tasks=16 Stdout:2025.04.02 03:32:14.411390 [ 5299 ] {} Context: Initialized background executor for common operations (e.g. clearing old parts) with num_threads=8, num_tasks=8 Stdout:2025.04.02 03:32:14.413544 [ 5299 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. Stdout:2025.04.02 03:32:14.413761 [ 5299 ] {} DNSCacheUpdater: Update period 15 seconds Stdout:2025.04.02 03:32:14.413778 [ 5299 ] {} Application: Loading metadata from /var/lib/clickhouse/ Stdout:2025.04.02 03:32:14.414441 [ 5316 ] {} DNSResolver: Updating DNS cache Stdout:2025.04.02 03:32:14.414475 [ 5316 ] {} DNSResolver: Updated DNS cache Stdout:2025.04.02 03:32:14.415211 [ 5299 ] {} Context: Database disk name: default Stdout:2025.04.02 03:32:14.415232 [ 5299 ] {} Context: Database disk name: default, path: /var/lib/clickhouse/ Stdout:2025.04.02 03:32:14.415263 [ 5299 ] {} loadSystemDatabase: metadata_file_path metadata/system.sql, existsFile true Stdout:2025.04.02 03:32:14.415354 [ 5299 ] {} loadSystemDatabase: metadata_file_path metadata/information_schema.sql, existsFile true Stdout:2025.04.02 03:32:14.415460 [ 5299 ] {} loadSystemDatabase: metadata_file_path metadata/INFORMATION_SCHEMA.sql, existsFile true Stdout:2025.04.02 03:32:14.415601 [ 5299 ] {} DatabaseAtomic (system): Metadata processed, database system has 0 tables and 0 dictionaries in total. Stdout:2025.04.02 03:32:14.415617 [ 5299 ] {} TablesLoader: Parsed metadata of 0 tables in 1 databases in 9.8525e-05 sec Stdout:2025.04.02 03:32:14.415622 [ 5299 ] {} ReferentialDeps: No tables Stdout:2025.04.02 03:32:14.415626 [ 5299 ] {} LoadingDeps: No tables Stdout:2025.04.02 03:32:14.415799 [ 5299 ] {} AsyncLoader: Prioritize load job 'startup Atomic database system': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:14.415808 [ 5299 ] {} AsyncLoader: Prioritize load job 'startup Ordinary database system': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:14.415818 [ 5299 ] {} AsyncLoader: Schedule load job 'startup Ordinary database system' into ForegroundLoad Stdout:2025.04.02 03:32:14.415824 [ 5299 ] {} AsyncLoader: Schedule load job 'startup Atomic database system' into ForegroundLoad Stdout:2025.04.02 03:32:14.415829 [ 5299 ] {} AsyncLoader: Change current priority: none -> 0 Stdout:2025.04.02 03:32:14.415834 [ 5299 ] {} AsyncLoader: Spawn loader worker #1 in ForegroundLoad Stdout:2025.04.02 03:32:14.415902 [ 5299 ] {} AsyncLoader: Wait load job 'startup Atomic database system' in ForegroundLoad Stdout:2025.04.02 03:32:14.416043 [ 5949 ] {} AsyncLoader: Execute load job 'startup Ordinary database system' in ForegroundLoad Stdout:2025.04.02 03:32:14.416081 [ 5949 ] {} AsyncLoader: Finish load job 'startup Ordinary database system' with status OK Stdout:2025.04.02 03:32:14.416089 [ 5949 ] {} AsyncLoader: Spawn loader worker #2 in ForegroundLoad Stdout:2025.04.02 03:32:14.416159 [ 5949 ] {} AsyncLoader: Execute load job 'startup Atomic database system' in ForegroundLoad Stdout:2025.04.02 03:32:14.416178 [ 5949 ] {} AsyncLoader: Finish load job 'startup Atomic database system' with status OK Stdout:2025.04.02 03:32:14.416185 [ 5949 ] {} AsyncLoader: Stop worker in ForegroundLoad Stdout:2025.04.02 03:32:14.416205 [ 5949 ] {} AsyncLoader: Stop worker in ForegroundLoad Stdout:2025.04.02 03:32:14.416209 [ 5949 ] {} AsyncLoader: Change current priority: 0 -> none Stdout:2025.04.02 03:32:14.416224 [ 5299 ] {} SystemLog: Creating system.query_log from query_log Stdout:2025.04.02 03:32:14.427264 [ 5299 ] {} SystemLog: Creating system.query_thread_log from query_thread_log Stdout:2025.04.02 03:32:14.427674 [ 5299 ] {} SystemLog: Creating system.part_log from part_log Stdout:2025.04.02 03:32:14.428004 [ 5299 ] {} SystemLog: Creating system.trace_log from trace_log Stdout:2025.04.02 03:32:14.428298 [ 5299 ] {} SystemLog: Creating system.crash_log from crash_log Stdout:2025.04.02 03:32:14.428442 [ 5299 ] {} SystemLog: Creating system.text_log from text_log Stdout:2025.04.02 03:32:14.428704 [ 5299 ] {} SystemLog: Creating system.metric_log from metric_log Stdout:2025.04.02 03:32:14.433541 [ 5299 ] {} SystemLog: Creating system.latency_log from latency_log Stdout:2025.04.02 03:32:14.433791 [ 5299 ] {} SystemLog: Creating system.error_log from error_log Stdout:2025.04.02 03:32:14.433992 [ 5299 ] {} SystemLog: Not creating system.filesystem_cache_log since corresponding section 'filesystem_cache_log' is missing from config Stdout:2025.04.02 03:32:14.434007 [ 5299 ] {} SystemLog: Not creating system.filesystem_read_prefetches_log since corresponding section 'filesystem_read_prefetches_log' is missing from config Executing query DROP TABLE test on node Stdout:2025.04.02 03:32:14.434029 [ 5299 ] {} SystemLog: Creating system.s3queue_log from s3queue_log Stdout:2025.04.02 03:32:14.434212 [ 5299 ] {} SystemLog: Not creating system.azure_queue_log since corresponding section 'azure_queue_log' is missing from config Stdout:2025.04.02 03:32:14.434226 [ 5299 ] {} SystemLog: Creating system.asynchronous_metric_log from asynchronous_metric_log Stdout:2025.04.02 03:32:14.434444 [ 5299 ] {} SystemLog: Creating system.opentelemetry_span_log from opentelemetry_span_log Stdout:2025.04.02 03:32:14.434690 [ 5299 ] {} SystemLog: Creating system.query_views_log from query_views_log Stdout:2025.04.02 03:32:14.435001 [ 5299 ] {} SystemLog: Not creating system.zookeeper_log since corresponding section 'zookeeper_log' is missing from config Stdout:2025.04.02 03:32:14.435033 [ 5299 ] {} SystemLog: Not creating system.session_log since corresponding section 'session_log' is missing from config Stdout:2025.04.02 03:32:14.435046 [ 5299 ] {} SystemLog: Not creating system.transactions_info_log since corresponding section 'transactions_info_log' is missing from config Stdout:2025.04.02 03:32:14.435059 [ 5299 ] {} SystemLog: Creating system.processors_profile_log from processors_profile_log Stdout:2025.04.02 03:32:14.435357 [ 5299 ] {} SystemLog: Creating system.asynchronous_insert_log from asynchronous_insert_log Stdout:2025.04.02 03:32:14.435666 [ 5299 ] {} SystemLog: Creating system.backup_log from backup_log Stdout:2025.04.02 03:32:14.436004 [ 5299 ] {} SystemLog: Creating system.blob_storage_log from blob_storage_log Stdout:2025.04.02 03:32:14.436266 [ 5299 ] {} SystemLog: Creating system.query_metric_log from query_metric_log Stdout:2025.04.02 03:32:14.441064 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.441081 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.442610 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.442627 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.442647 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.442651 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.442667 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.442670 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.442728 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.442735 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.442780 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.442785 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.442907 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.442914 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.442961 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.442965 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443044 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443051 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443100 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443105 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443141 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443145 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443183 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443187 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443262 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443268 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443302 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443361 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443419 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443424 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443459 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443464 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443486 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443489 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443510 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443515 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443543 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443547 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443564 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443567 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443590 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443593 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443614 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443618 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443635 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443639 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443666 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443670 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443687 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443691 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443748 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443752 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443782 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443786 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.443965 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.443973 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444009 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444013 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444047 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444051 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444074 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444078 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444108 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444112 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444155 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444159 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444194 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444200 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444248 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444256 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444346 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444353 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444437 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444442 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444515 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444519 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444558 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444562 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444698 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444705 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444733 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444739 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444760 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444764 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444788 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444792 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444825 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444830 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444850 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444854 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444870 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444875 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444924 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444929 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444963 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.444967 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.444998 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.445005 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.445233 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.445241 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.445336 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.445354 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.445420 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.445425 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.445449 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.445452 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.445492 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.445496 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.445579 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.445585 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.445607 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.445612 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.445629 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.445633 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.445863 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.445871 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446009 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446015 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446088 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446093 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446235 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446245 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446351 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446356 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446406 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446410 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446449 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446453 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446596 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446602 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446663 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446674 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446727 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446733 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446775 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446784 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.446855 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.446864 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447060 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447069 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447129 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447133 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447192 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447197 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447228 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447231 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447298 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447303 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447335 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447338 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447391 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447395 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447448 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447456 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447486 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447490 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447523 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447527 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447575 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447584 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447663 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447673 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447747 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447762 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447856 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447867 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447914 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447924 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447956 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.447960 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.447996 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448000 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448046 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448051 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448106 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448111 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448184 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448190 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448218 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448222 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448255 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448259 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448284 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448287 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448315 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448319 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448339 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448342 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448364 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448367 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448418 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448422 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448447 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448451 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448473 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448476 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448515 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448519 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448576 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448581 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.448676 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.448682 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.456465 [ 5299 ] {} DatabaseCatalog: Found 6 partially dropped tables. Will load them and retry removal. Stdout:2025.04.02 03:32:14.456837 [ 5971 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3) from metadata_dropped/default.test_add_disk.8a487b06-210e-4c1c-91cf-235fb0fcffb3.sql Stdout:2025.04.02 03:32:14.456937 [ 5970 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174) from metadata_dropped/default.test_add_disk.ab1b9e7f-3e87-42e0-987c-542af287b174.sql Stdout:2025.04.02 03:32:14.457249 [ 5972 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f) from metadata_dropped/default.test_add_disk_to_policy.e1af3e60-fe6e-4895-a02b-fced2189743f.sql Stdout:2025.04.02 03:32:14.457446 [ 5974 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c) from metadata_dropped/default.test_add_policy.d69b6b75-0af1-4149-a0f9-c13bb05ea67c.sql Stdout:2025.04.02 03:32:14.457521 [ 5971 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:14.457597 [ 5970 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:14.457612 [ 5972 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:14.457908 [ 5973 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5) from metadata_dropped/default.test_add_volume_to_policy.b03aef8f-055b-454d-bac3-7055c6fd12b5.sql Stdout:2025.04.02 03:32:14.457980 [ 5975 ] {} DatabaseCatalog: Trying load partially dropped table default.test_new_policy_works (9288e96b-23dc-4115-80b9-90a084e01018) from metadata_dropped/default.test_new_policy_works.9288e96b-23dc-4115-80b9-90a084e01018.sql Stdout:2025.04.02 03:32:14.458074 [ 5974 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:14.458313 [ 5973 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:14.459350 [ 5975 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:14.459858 [ 5970 ] {} StoragePolicy (small_jbod_with_external): Storage policy small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:14.459873 [ 5970 ] {} StoragePolicySelector: Storage policy `small_jbod_with_external` loaded Stdout:2025.04.02 03:32:14.460038 [ 5970 ] {} StoragePolicy (one_more_small_jbod_with_external): Storage policy one_more_small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:14.460051 [ 5970 ] {} StoragePolicySelector: Storage policy `one_more_small_jbod_with_external` loaded Stdout:2025.04.02 03:32:14.460218 [ 5970 ] {} StoragePolicy (jbods_with_external): Storage policy jbods_with_external created, total volumes 2 Stdout:2025.04.02 03:32:14.460226 [ 5970 ] {} StoragePolicySelector: Storage policy `jbods_with_external` loaded Stdout:2025.04.02 03:32:14.460345 [ 5970 ] {} StoragePolicy (moving_jbod_with_external): Storage policy moving_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:14.460352 [ 5970 ] {} StoragePolicySelector: Storage policy `moving_jbod_with_external` loaded Stdout:2025.04.02 03:32:14.460412 [ 5970 ] {} StorageConfiguration: Volume `small` max_data_part_size is too low (2.00 MiB < 8.00 MiB) Stdout:2025.04.02 03:32:14.460492 [ 5970 ] {} StoragePolicy (default_disk_with_external): Storage policy default_disk_with_external created, total volumes 2 Stdout:2025.04.02 03:32:14.460496 [ 5970 ] {} StoragePolicySelector: Storage policy `default_disk_with_external` loaded Stdout:2025.04.02 03:32:14.460677 [ 5970 ] {} StorageConfiguration: Volume `special_warning_small_volume` max_data_part_size is too low (1.00 KiB < 8.00 MiB) Stdout:2025.04.02 03:32:14.460763 [ 5970 ] {} StoragePolicy (special_warning_policy): Storage policy special_warning_policy created, total volumes 4 Stdout:2025.04.02 03:32:14.460767 [ 5970 ] {} StoragePolicySelector: Storage policy `special_warning_policy` loaded Stdout:2025.04.02 03:32:14.460782 [ 5970 ] {} StoragePolicy (default): Storage policy default created, total volumes 1 Stdout:2025.04.02 03:32:14.460897 [ 5970 ] {} default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174): Loading data parts Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:2025.04.02 03:32:14.460994 [ 5971 ] {} default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3): Loading data parts Stdout:2025.04.02 03:32:14.461574 [ 5975 ] {} DatabaseCatalog: Cannot load partially dropped table default.test_new_policy_works (9288e96b-23dc-4115-80b9-90a084e01018) from: metadata_dropped/default.test_new_policy_works.9288e96b-23dc-4115-80b9-90a084e01018.sql. Parsed query: ATTACH TABLE default.test_new_policy_works UUID '9288e96b-23dc-4115-80b9-90a084e01018' (`d` UInt64) ENGINE = MergeTree ORDER BY d SETTINGS storage_policy = 'cool_policy', index_granularity = 8192. Will remove metadata and store/928/9288e96b-23dc-4115-80b9-90a084e01018/. Garbage may be left in ZooKeeper.: Code: 478. DB::Exception: Unknown storage policy `cool_policy`. (UNKNOWN_POLICY), Stack trace (when copying this message, always include the lines below): Stdout: Stdout:0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000cfb8fbb Stdout:1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007ff830c Stdout:2. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x0000000007ff802b Stdout:3. DB::StoragePolicySelector::get(String const&) const @ 0x00000000105c18e1 Stdout:4. DB::Context::getStoragePolicy(String const&) const @ 0x0000000010b34cbf Stdout:5. DB::MergeTreeData::getStoragePolicy() const @ 0x000000001201df88 Stdout:6. DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x00000000120136a8 Stdout:7. DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x00000000123e6548 Stdout:8. DB::create(DB::StorageFactory::Arguments const&) @ 0x00000000123e2c8c Stdout:9. DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x00000000119e37bf Stdout:10. DB::createTableFromAST(DB::ASTCreateQuery, String const&, String const&, std::shared_ptr, DB::LoadingStrictnessLevel) @ 0x00000000104454aa Stdout:11. DB::DatabaseCatalog::enqueueDroppedTableCleanup(DB::StorageID, std::shared_ptr, String, bool) @ 0x0000000010c038a9 Stdout:12. void std::__function::__policy_invoker::__call_impl[abi:ne190107]>(std::__function::__policy_storage const*) @ 0x0000000010c14b0e Stdout:13. void DB::ThreadPoolCallbackRunnerLocal>, std::function>::executeCallback>(std::promise&, std::function&&) @ 0x000000000efb3e95 Stdout:14. DB::ThreadPoolCallbackRunnerLocal>, std::function>::operator()(std::function&&, Priority, std::optional)::'lambda'()::operator()() @ 0x000000000efb3c9b Stdout:15. ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x000000000d0992db Stdout:16. void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl>::ThreadFromThreadPool::*)(), ThreadPoolImpl>::ThreadFromThreadPool*>(void (ThreadPoolImpl>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl>::ThreadFromThreadPool*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000d09fda3 Stdout:17. ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000000d09656f Stdout:18. void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000000d09d85a Stdout:19. ? @ 0x00007fb1b4723ac3 Stdout:20. ? @ 0x00007fb1b47b5850 Stdout: (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:14.461794 [ 5973 ] {} default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5): Loading data parts Stdout:2025.04.02 03:32:14.461803 [ 5972 ] {} default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f): Loading data parts Stdout:2025.04.02 03:32:14.462609 [ 5973 ] {} default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5): There are no data parts Stdout:2025.04.02 03:32:14.462724 [ 5974 ] {} default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c): Loading data parts Stdout:2025.04.02 03:32:14.470925 [ 5971 ] {} default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3): There are no data parts Stdout:2025.04.02 03:32:14.470999 [ 5974 ] {} default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c): There are no data parts Stdout:2025.04.02 03:32:14.471002 [ 5972 ] {} default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f): There are no data parts Stdout:2025.04.02 03:32:14.471309 [ 5970 ] {} default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174): There are no data parts Stdout:2025.04.02 03:32:14.471607 [ 5299 ] {} DatabaseAtomic (default): Metadata processed, database default has 0 tables and 0 dictionaries in total. Stdout:2025.04.02 03:32:14.471628 [ 5299 ] {} TablesLoader: Parsed metadata of 0 tables in 1 databases in 5.4422e-05 sec Stdout:2025.04.02 03:32:14.471634 [ 5299 ] {} ReferentialDeps: No tables Stdout:2025.04.02 03:32:14.471638 [ 5299 ] {} LoadingDeps: No tables Stdout:2025.04.02 03:32:14.471663 [ 5299 ] {} loadMetadata: Start asynchronous loading of databases Stdout:2025.04.02 03:32:14.471672 [ 5299 ] {} AsyncLoader: Schedule load job 'startup Ordinary database default' into BackgrndStartup Stdout:2025.04.02 03:32:14.471677 [ 5299 ] {} AsyncLoader: Schedule load job 'startup Atomic database default' into BackgrndStartup Executing query SYSTEM START FETCHES limited_sends2 on node1 Stdout:2025.04.02 03:32:14.471682 [ 5299 ] {} AsyncLoader: Change current priority: none -> 2 Stdout:2025.04.02 03:32:14.471692 [ 5299 ] {} AsyncLoader: Spawn loader worker #1 in BackgrndStartup Stdout:2025.04.02 03:32:14.471733 [ 5299 ] {} UserDefinedSQLObjectsLoaderFromDisk: Loading user defined objects from /var/lib/clickhouse/user_defined/ Stdout:2025.04.02 03:32:14.471740 [ 5299 ] {} UserDefinedSQLObjectsLoaderFromDisk: The directory for user defined objects (/var/lib/clickhouse/user_defined/) does not exist: nothing to load Stdout:2025.04.02 03:32:14.471741 [ 5982 ] {} AsyncLoader: Execute load job 'startup Ordinary database default' in BackgrndStartup Stdout:2025.04.02 03:32:14.471785 [ 5299 ] {} WorkloadEntityDiskStorage: Loading workload entities from /var/lib/clickhouse/workload/ Stdout:2025.04.02 03:32:14.471801 [ 5299 ] {} WorkloadEntityDiskStorage: The directory for workload entities (/var/lib/clickhouse/workload/) does not exist: nothing to load Stdout:2025.04.02 03:32:14.471788 [ 5982 ] {} AsyncLoader: Finish load job 'startup Ordinary database default' with status OK Stdout:2025.04.02 03:32:14.471818 [ 5982 ] {} AsyncLoader: Spawn loader worker #2 in BackgrndStartup Stdout:2025.04.02 03:32:14.471830 [ 5982 ] {} AsyncLoader: Execute load job 'startup Atomic database default' in BackgrndStartup Stdout:2025.04.02 03:32:14.471838 [ 5982 ] {} AsyncLoader: Finish load job 'startup Atomic database default' with status OK Stdout:2025.04.02 03:32:14.471845 [ 5982 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:14.471854 [ 5982 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:14.471858 [ 5982 ] {} AsyncLoader: Change current priority: 2 -> none Stdout:2025.04.02 03:32:14.472377 [ 5299 ] {} ZooKeeperClient: Adding ZooKeeper host zoo3:2181 (172.16.8.3:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:14.472795 [ 5299 ] {} ZooKeeperClient: Adding ZooKeeper host zoo1:2181 (172.16.8.2:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:14.473105 [ 5299 ] {} ZooKeeperClient: Adding ZooKeeper host zoo2:2181 (172.16.8.4:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:14.474420 [ 5299 ] {} ZooKeeperClient: Connected to ZooKeeper at 172.16.8.3:2181 with session_id 12 Stdout:2025.04.02 03:32:14.474686 [ 5299 ] {} ZooKeeperClient: Keeper feature flag FILTERED_LIST: disabled Stdout:2025.04.02 03:32:14.474693 [ 5299 ] {} ZooKeeperClient: Keeper feature flag MULTI_READ: disabled Stdout:2025.04.02 03:32:14.474700 [ 5299 ] {} ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: disabled Stdout:2025.04.02 03:32:14.474703 [ 5299 ] {} ZooKeeperClient: Keeper feature flag CREATE_IF_NOT_EXISTS: disabled Stdout:2025.04.02 03:32:14.474705 [ 5299 ] {} ZooKeeperClient: Keeper feature flag REMOVE_RECURSIVE: disabled Stdout:2025.04.02 03:32:14.474710 [ 5299 ] {} ZooKeeper: Initialized, hosts: zoo1:2181,zoo2:2181,zoo3:2181 Stdout:2025.04.02 03:32:14.475654 [ 5299 ] {} Application: Loaded metadata. Stdout:2025.04.02 03:32:14.475755 [ 5299 ] {} bool DB::(anonymous namespace)::checkPermissionsImpl(): Code: 412. DB::Exception: Can't receive Netlink response: error -2. (NETLINK_ERROR) (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:14.475774 [ 5299 ] {} Application: Tasks stats provider: procfs Stdout:2025.04.02 03:32:14.475839 [ 5299 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:14.475853 [ 5299 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:14.476095 [ 5299 ] {} MySQLHandlerFactory: Failed to create SSL context. SSL will be disabled. Error: Poco::Exception. Code: 1000, e.code() = 0, SSL Exception: Configuration error: no certificate file has been specified (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:14.476134 [ 5299 ] {} MySQLHandlerFactory: Failed to read RSA key pair from server certificate. Error: Code: 139. DB::Exception: Certificate file is not set. (NO_ELEMENTS_IN_CONFIG) (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:14.476139 [ 5299 ] {} MySQLHandlerFactory: Generating new RSA key pair. Stdout:2025.04.02 03:32:14.522505 [ 5299 ] {} CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. Stdout:2025.04.02 03:32:14.522526 [ 5299 ] {} CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. Stdout:2025.04.02 03:32:14.525413 [ 5299 ] {} AsyncLoader: Schedule load job 'startup ddl worker' into BackgrndStartup Stdout:2025.04.02 03:32:14.525429 [ 5299 ] {} AsyncLoader: Change current priority: none -> 2 Stdout:2025.04.02 03:32:14.525434 [ 5299 ] {} AsyncLoader: Spawn loader worker #1 in BackgrndStartup Stdout:2025.04.02 03:32:14.525543 [ 5299 ] {} Application: Listening for http://0.0.0.0:8123 Stdout:2025.04.02 03:32:14.525564 [ 5974 ] {} AsyncLoader: Execute load job 'startup ddl worker' in BackgrndStartup Stdout:2025.04.02 03:32:14.525596 [ 5299 ] {} Application: Listening for native protocol (tcp): 0.0.0.0:9000 Stdout:2025.04.02 03:32:14.525606 [ 5974 ] {} AsyncLoader: Finish load job 'startup ddl worker' with status OK Stdout:2025.04.02 03:32:14.525619 [ 5974 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:14.525626 [ 5974 ] {} AsyncLoader: Change current priority: 2 -> none Stdout:2025.04.02 03:32:14.525638 [ 5974 ] {} DDLWorker: Started DDLWorker cleanup thread Stdout:2025.04.02 03:32:14.525643 [ 5299 ] {} Application: Listening for MySQL compatibility protocol: 0.0.0.0:9004 Stdout:2025.04.02 03:32:14.525618 [ 5981 ] {} DDLWorker: Starting DDLWorker thread Stdout:2025.04.02 03:32:14.525665 [ 5981 ] {} DDLWorker: Initializing DDLWorker thread Stdout:2025.04.02 03:32:14.525686 [ 5299 ] {} Application: Listening for PostgreSQL compatibility protocol: 0.0.0.0:9005 Stdout:2025.04.02 03:32:14.525694 [ 5299 ] {} Application: Ready for connections. Stdout:2025.04.02 03:32:14.533188 [ 5981 ] {} DDLWorker: Initialized DDLWorker thread Stdout:2025.04.02 03:32:14.533442 [ 5981 ] {} DDLWorker: Scheduling tasks Stdout:2025.04.02 03:32:14.533450 [ 5981 ] {} DDLWorker: Don't have unfinished tasks after restarting Stdout:2025.04.02 03:32:14.533628 [ 5981 ] {} DDLWorker: scheduleTasks: initialized=true, size_before_filtering=0, queue_size=0, entries=none..none, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=none Stdout:2025.04.02 03:32:14.533634 [ 5981 ] {} DDLWorker: No tasks to schedule Stdout:2025.04.02 03:32:14.533639 [ 5981 ] {} DDLWorker: Waiting for queue updates Stdout:2025.04.02 03:32:14.534031 [ 5974 ] {} DDLWorker: Cleaning queue Stdout:2025.04.02 03:32:15.186834 [ 5312 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57412 Stdout:2025.04.02 03:32:15.186911 [ 5312 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:15.186931 [ 5312 ] {} TCP-Session-eb82764d-817b-4ce3-95f5-41cffea7f5e8: Authenticating user 'default' from 172.16.8.1:57412 Stdout:2025.04.02 03:32:15.186967 [ 5312 ] {} TCP-Session-eb82764d-817b-4ce3-95f5-41cffea7f5e8: eb82764d-817b-4ce3-95f5-41cffea7f5e8 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:15.186980 [ 5312 ] {} TCP-Session-eb82764d-817b-4ce3-95f5-41cffea7f5e8: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:15.187076 [ 5312 ] {} ContextAccess (default): Settings: readonly = 0, allow_ddl = true, allow_introspection_functions = false Stdout:2025.04.02 03:32:15.187102 [ 5312 ] {} ContextAccess (default): List of all grants: GRANT CHECK, SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, ROLE ADMIN, CREATE ROW POLICY, ALTER ROW POLICY, DROP ROW POLICY, CREATE QUOTA, ALTER QUOTA, DROP QUOTA, CREATE SETTINGS PROFILE, ALTER SETTINGS PROFILE, DROP SETTINGS PROFILE, ALLOW SQL SECURITY NONE, SHOW ACCESS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* WITH GRANT OPTION, GRANT TABLE ENGINE ON * WITH GRANT OPTION, GRANT CREATE USER, ALTER USER, DROP USER, CREATE ROLE, ALTER ROLE, DROP ROLE, SET DEFINER ON * WITH GRANT OPTION, GRANT ALTER NAMED COLLECTION, CREATE NAMED COLLECTION, DROP NAMED COLLECTION, SHOW NAMED COLLECTIONS, NAMED COLLECTION ON * WITH GRANT OPTION Stdout:2025.04.02 03:32:15.187113 [ 5312 ] {} ContextAccess (default): List of all grants including implicit: GRANT CHECK, SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, ROLE ADMIN, CREATE ROW POLICY, ALTER ROW POLICY, DROP ROW POLICY, CREATE QUOTA, ALTER QUOTA, DROP QUOTA, CREATE SETTINGS PROFILE, ALTER SETTINGS PROFILE, DROP SETTINGS PROFILE, ALLOW SQL SECURITY NONE, SHOW ACCESS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* WITH GRANT OPTION, GRANT TABLE ENGINE ON * WITH GRANT OPTION, GRANT CREATE USER, ALTER USER, DROP USER, CREATE ROLE, ALTER ROLE, DROP ROLE, SET DEFINER ON * WITH GRANT OPTION, GRANT ALTER NAMED COLLECTION, CREATE NAMED COLLECTION, DROP NAMED COLLECTION, SHOW NAMED COLLECTIONS, NAMED COLLECTION ON * WITH GRANT OPTION Stdout:2025.04.02 03:32:15.196000 [ 5312 ] {} TCP-Session-eb82764d-817b-4ce3-95f5-41cffea7f5e8: eb82764d-817b-4ce3-95f5-41cffea7f5e8 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:15.196302 [ 5312 ] {e3476711-55d2-4c32-9050-dc2e90a82370} executeQuery: (from 172.16.8.1:57412) (query 1, line 1) select 20 (stage: Complete) Stdout:2025.04.02 03:32:15.196429 [ 5312 ] {e3476711-55d2-4c32-9050-dc2e90a82370} Planner: Query to stage Complete Stdout:2025.04.02 03:32:15.196481 [ 5312 ] {e3476711-55d2-4c32-9050-dc2e90a82370} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:15.197179 [ 5312 ] {e3476711-55d2-4c32-9050-dc2e90a82370} executeQuery: Read 1 rows, 1.00 B in 0.000879 sec., 1137.6564277588168 rows/sec., 1.11 KiB/sec. Stdout:2025.04.02 03:32:15.197268 [ 5312 ] {e3476711-55d2-4c32-9050-dc2e90a82370} TCPHandler: Processed in 0.001367094 sec. Stdout:2025.04.02 03:32:15.207429 [ 5312 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:15.207445 [ 5312 ] {} TCP-Session-eb82764d-817b-4ce3-95f5-41cffea7f5e8: eb82764d-817b-4ce3-95f5-41cffea7f5e8 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.254043 [ 5312 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57420 Stdout:2025.04.02 03:32:17.254086 [ 5312 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:17.254101 [ 5312 ] {} TCP-Session-ceb5f718-3a95-434a-8a0d-57fddbad32f8: Authenticating user 'default' from 172.16.8.1:57420 Stdout:2025.04.02 03:32:17.254125 [ 5312 ] {} TCP-Session-ceb5f718-3a95-434a-8a0d-57fddbad32f8: ceb5f718-3a95-434a-8a0d-57fddbad32f8 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.254130 [ 5312 ] {} TCP-Session-ceb5f718-3a95-434a-8a0d-57fddbad32f8: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.263070 [ 5312 ] {} TCP-Session-ceb5f718-3a95-434a-8a0d-57fddbad32f8: ceb5f718-3a95-434a-8a0d-57fddbad32f8 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:17.263275 [ 5312 ] {30253f31-18e1-42a1-b887-b7a1d64e487f} executeQuery: (from 172.16.8.1:57420) (query 1, line 2) CREATE TABLE test_remove_disk ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' (stage: Complete) Stdout:2025.04.02 03:32:17.263534 [ 5312 ] {30253f31-18e1-42a1-b887-b7a1d64e487f} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:17.264051 [ 5312 ] {30253f31-18e1-42a1-b887-b7a1d64e487f} default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce): Loading data parts Stdout:2025.04.02 03:32:17.264373 [ 5312 ] {30253f31-18e1-42a1-b887-b7a1d64e487f} default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce): There are no data parts Stdout:2025.04.02 03:32:17.264443 [ 5312 ] {30253f31-18e1-42a1-b887-b7a1d64e487f} AsyncLoader: Prioritize load job 'startup Atomic database default': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:17.264450 [ 5312 ] {30253f31-18e1-42a1-b887-b7a1d64e487f} AsyncLoader: Prioritize load job 'startup Ordinary database default': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:17.266037 [ 5312 ] {30253f31-18e1-42a1-b887-b7a1d64e487f} DatabaseAtomic (default): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:17.266049 [ 5312 ] {30253f31-18e1-42a1-b887-b7a1d64e487f} DatabaseAtomic (default): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:17.266319 [ 5312 ] {30253f31-18e1-42a1-b887-b7a1d64e487f} TCPHandler: Processed in 0.003352717 sec. Stdout:2025.04.02 03:32:17.266313 [ 5318 ] {} default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce): Didn't start merge: There are no parts that can be merged. (Collector returned empty ranges set) Stdout:2025.04.02 03:32:17.269805 [ 5312 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:17.269984 [ 5312 ] {} TCP-Session-ceb5f718-3a95-434a-8a0d-57fddbad32f8: ceb5f718-3a95-434a-8a0d-57fddbad32f8 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.320525 [ 5312 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57422 Stdout:2025.04.02 03:32:17.320569 [ 5312 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:17.320584 [ 5312 ] {} TCP-Session-56279c18-b52c-41aa-8f40-10881e0b6fff: Authenticating user 'default' from 172.16.8.1:57422 Stdout:2025.04.02 03:32:17.320607 [ 5312 ] {} TCP-Session-56279c18-b52c-41aa-8f40-10881e0b6fff: 56279c18-b52c-41aa-8f40-10881e0b6fff Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.320613 [ 5312 ] {} TCP-Session-56279c18-b52c-41aa-8f40-10881e0b6fff: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.330184 [ 5312 ] {} TCP-Session-56279c18-b52c-41aa-8f40-10881e0b6fff: 56279c18-b52c-41aa-8f40-10881e0b6fff Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:17.330395 [ 5312 ] {fdaa7cff-6ae8-4d1f-a730-f4e10a630bee} executeQuery: (from 172.16.8.1:57422) (query 1, line 1) SELECT name FROM system.disks (stage: Complete) Stdout:2025.04.02 03:32:17.330555 [ 5312 ] {fdaa7cff-6ae8-4d1f-a730-f4e10a630bee} Planner: Query to stage Complete Stdout:2025.04.02 03:32:17.330708 [ 5312 ] {fdaa7cff-6ae8-4d1f-a730-f4e10a630bee} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:17.331446 [ 5312 ] {fdaa7cff-6ae8-4d1f-a730-f4e10a630bee} executeQuery: Read 5 rows, 613.00 B in 0.001078 sec., 4638.21892393321 rows/sec., 555.32 KiB/sec. Stdout:2025.04.02 03:32:17.331538 [ 5312 ] {fdaa7cff-6ae8-4d1f-a730-f4e10a630bee} TCPHandler: Processed in 0.001555797 sec. Stdout:2025.04.02 03:32:17.341955 [ 5312 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:17.341996 [ 5312 ] {} TCP-Session-56279c18-b52c-41aa-8f40-10881e0b6fff: 56279c18-b52c-41aa-8f40-10881e0b6fff Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.392419 [ 5312 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57434 Stdout:2025.04.02 03:32:17.392466 [ 5312 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:17.392481 [ 5312 ] {} TCP-Session-a5593f17-b440-4a27-a0dd-e41454f4805c: Authenticating user 'default' from 172.16.8.1:57434 Stdout:2025.04.02 03:32:17.392503 [ 5312 ] {} TCP-Session-a5593f17-b440-4a27-a0dd-e41454f4805c: a5593f17-b440-4a27-a0dd-e41454f4805c Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.392509 [ 5312 ] {} TCP-Session-a5593f17-b440-4a27-a0dd-e41454f4805c: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.402369 [ 5312 ] {} TCP-Session-a5593f17-b440-4a27-a0dd-e41454f4805c: a5593f17-b440-4a27-a0dd-e41454f4805c Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:17.402624 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} executeQuery: (from 172.16.8.1:57434) (query 1, line 1) SYSTEM RELOAD CONFIG (stage: Complete) Stdout:2025.04.02 03:32:17.410208 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigReloader: Loading config '/etc/clickhouse-server/config.xml' Stdout:2025.04.02 03:32:17.410226 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/config.xml'. Stdout:2025.04.02 03:32:17.411119 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:17.411205 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:17.411281 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_enable_keeper_async_replication.xml'. Stdout:2025.04.02 03:32:17.411336 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_instance_config.xml'. Stdout:2025.04.02 03:32:17.411900 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/logs_config.xml'. Stdout:2025.04.02 03:32:17.412003 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/storage_configuration.xml'. Stdout:2025.04.02 03:32:17.416407 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'. Stdout:2025.04.02 03:32:17.416502 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performing update on configuration Stdout:2025.04.02 03:32:17.423799 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} Application: Changed setting 'max_server_memory_usage' to 27.54 GiB (30.60 GiB available memory * 0.90 max_server_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:17.423813 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.30 GiB (30.60 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:17.423823 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} Application: Merges and mutations memory limit is set to 15.30 GiB Stdout:2025.04.02 03:32:17.424734 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} Application: Setting max_remote_read_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:17.424744 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} Application: Setting max_remote_write_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:17.424749 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} Application: ConcurrencyControl limit is set to 32 CPU slots with 'fair_round_robin' scheduler Stdout:2025.04.02 03:32:17.424761 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} MergeTreeBackgroundExecutor: Loaded new threads count (16) and max tasks count (32) for MergeMutateExecutor Stdout:2025.04.02 03:32:17.424767 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for MoveExecutor Stdout:2025.04.02 03:32:17.424771 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} MergeTreeBackgroundExecutor: Loaded new threads count (16) and max tasks count (16) for FetchExecutor Stdout:2025.04.02 03:32:17.424779 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for CommonExecutor Stdout:2025.04.02 03:32:17.425140 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} DiskSelector: Disk `remove_disk_jbod3` disappeared from configuration, this change will be applied after restart of ClickHouse Stdout:2025.04.02 03:32:17.425381 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (small_jbod_with_external): Storage policy small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:17.425387 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicySelector: Storage policy `small_jbod_with_external` loaded Stdout:2025.04.02 03:32:17.425494 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (one_more_small_jbod_with_external): Storage policy one_more_small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:17.425499 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicySelector: Storage policy `one_more_small_jbod_with_external` loaded Stdout:2025.04.02 03:32:17.425621 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (jbods_with_external): Storage policy jbods_with_external created, total volumes 2 Stdout:2025.04.02 03:32:17.425626 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicySelector: Storage policy `jbods_with_external` loaded Stdout:2025.04.02 03:32:17.425737 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (moving_jbod_with_external): Storage policy moving_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:17.425741 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicySelector: Storage policy `moving_jbod_with_external` loaded Stdout:2025.04.02 03:32:17.425803 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StorageConfiguration: Volume `small` max_data_part_size is too low (2.00 MiB < 8.00 MiB) Stdout:2025.04.02 03:32:17.425881 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (default_disk_with_external): Storage policy default_disk_with_external created, total volumes 2 Stdout:2025.04.02 03:32:17.425886 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicySelector: Storage policy `default_disk_with_external` loaded Stdout:2025.04.02 03:32:17.426059 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StorageConfiguration: Volume `special_warning_small_volume` max_data_part_size is too low (1.00 KiB < 8.00 MiB) Stdout:2025.04.02 03:32:17.426139 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (special_warning_policy): Storage policy special_warning_policy created, total volumes 4 Stdout:2025.04.02 03:32:17.426144 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicySelector: Storage policy `special_warning_policy` loaded Stdout:2025.04.02 03:32:17.426168 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (default): Storage policy default created, total volumes 1 Stdout:2025.04.02 03:32:17.426201 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (default): Storage policy default created, total volumes 1 Stdout:2025.04.02 03:32:17.426367 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StorageConfiguration: Volume `small` max_data_part_size is too low (2.00 MiB < 8.00 MiB) Stdout:2025.04.02 03:32:17.426452 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (default_disk_with_external): Storage policy default_disk_with_external created, total volumes 2 Stdout:2025.04.02 03:32:17.426497 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StorageConfiguration: Volume `small` max_data_part_size is too low (2.00 MiB < 8.00 MiB) Stdout:2025.04.02 03:32:17.426687 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (jbods_with_external): Storage policy jbods_with_external created, total volumes 2 Stdout:2025.04.02 03:32:17.426899 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (moving_jbod_with_external): Storage policy moving_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:17.427115 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (one_more_small_jbod_with_external): Storage policy one_more_small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:17.427327 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (small_jbod_with_external): Storage policy small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:17.427598 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StorageConfiguration: Volume `special_warning_small_volume` max_data_part_size is too low (1.00 KiB < 8.00 MiB) Stdout:2025.04.02 03:32:17.427684 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StoragePolicy (special_warning_policy): Storage policy special_warning_policy created, total volumes 4 Stdout:2025.04.02 03:32:17.427868 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} StorageConfiguration: Volume `special_warning_small_volume` max_data_part_size is too low (1.00 KiB < 8.00 MiB) Stdout:2025.04.02 03:32:17.428099 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration Stdout:2025.04.02 03:32:17.429806 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigReloader: Loading config '/etc/clickhouse-server/users.xml' Stdout:2025.04.02 03:32:17.434092 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'. Stdout:2025.04.02 03:32:17.434174 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performing update on configuration Stdout:2025.04.02 03:32:17.434951 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performed update on configuration Stdout:2025.04.02 03:32:17.435665 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} MemoryTracker: Query peak memory usage: 10.28 MiB. Stdout:2025.04.02 03:32:17.435751 [ 5312 ] {628213b6-db17-4ac6-86ee-bf8692227f42} TCPHandler: Processed in 0.033545281 sec. Stdout:2025.04.02 03:32:17.436464 [ 5312 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:17.436497 [ 5312 ] {} TCP-Session-a5593f17-b440-4a27-a0dd-e41454f4805c: a5593f17-b440-4a27-a0dd-e41454f4805c Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.507100 [ 5312 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57436 Stdout:2025.04.02 03:32:17.507139 [ 5312 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:17.507156 [ 5312 ] {} TCP-Session-c5e25c6f-fbd5-4aec-acdb-83f345197a0b: Authenticating user 'default' from 172.16.8.1:57436 Stdout:2025.04.02 03:32:17.507179 [ 5312 ] {} TCP-Session-c5e25c6f-fbd5-4aec-acdb-83f345197a0b: c5e25c6f-fbd5-4aec-acdb-83f345197a0b Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.507186 [ 5312 ] {} TCP-Session-c5e25c6f-fbd5-4aec-acdb-83f345197a0b: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:17.519347 [ 5312 ] {} TCP-Session-c5e25c6f-fbd5-4aec-acdb-83f345197a0b: c5e25c6f-fbd5-4aec-acdb-83f345197a0b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:17.519572 [ 5312 ] {d42d0bd7-cf9e-451d-9e99-62ed73cf905e} executeQuery: (from 172.16.8.1:57436) (query 1, line 1) SELECT name FROM system.disks (stage: Complete) Stdout:2025.04.02 03:32:17.519736 [ 5312 ] {d42d0bd7-cf9e-451d-9e99-62ed73cf905e} Planner: Query to stage Complete Stdout:2025.04.02 03:32:17.519880 [ 5312 ] {d42d0bd7-cf9e-451d-9e99-62ed73cf905e} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:17.520544 [ 5312 ] {d42d0bd7-cf9e-451d-9e99-62ed73cf905e} executeQuery: Read 5 rows, 613.00 B in 0.001007 sec., 4965.243296921549 rows/sec., 594.47 KiB/sec. Stdout:2025.04.02 03:32:17.520639 [ 5312 ] {d42d0bd7-cf9e-451d-9e99-62ed73cf905e} TCPHandler: Processed in 0.001404864 sec. Stdout:2025.04.02 03:32:17.530834 [ 5312 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:17.530858 [ 5312 ] {} TCP-Session-c5e25c6f-fbd5-4aec-acdb-83f345197a0b: c5e25c6f-fbd5-4aec-acdb-83f345197a0b Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Executing query DROP TABLE IF EXISTS test_remove_disk on node1 Executing query SYSTEM START FETCHES limited_sends3 on node1 Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Running Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Running Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Running Stderr: Container roottestreloadzookeeper-gw5-node-1 Creating Stderr: Container roottestreloadzookeeper-gw5-node-1 Created Stderr: Container roottestreloadzookeeper-gw5-node-1 Starting Stderr: Container roottestreloadzookeeper-gw5-node-1 Started ClickHouse instance created get_instance_ip instance_name=node http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-node-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-node-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node, ip: 172.16.5.5... http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-node-1/json HTTP/1.1" 200 None Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 http://localhost:None "GET /v1.46/containers/4be7578f3f1d21e99da31eaf1ef6dc71af266c20fb8e73c115f8921f6d0834f0/json HTTP/1.1" 200 None [gw2] PASSED test_reloading_storage_configuration/test.py::test_remove_disk test_reloading_storage_configuration/test.py::test_remove_disk_from_policy run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/.env --project-name roottestreloadmaxtablesizetodrop-gw7 --file /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/node/docker-compose.yml stop --timeout 20] [gw7] PASSED test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout: PID TTY TIME CMD Stdout: 5299 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] Executing query SYSTEM START FETCHES limited_sends4 on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] http://localhost:None "GET /v1.46/containers/4be7578f3f1d21e99da31eaf1ef6dc71af266c20fb8e73c115f8921f6d0834f0/json HTTP/1.1" 200 None Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:5299 Stdout:12097047 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:234620450 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] http://localhost:None "GET /v1.46/containers/4be7578f3f1d21e99da31eaf1ef6dc71af266c20fb8e73c115f8921f6d0834f0/json HTTP/1.1" 200 None Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query ALTER TABLE test_rename_parallel_same_node RENAME COLUMN foo2 to num2 on node1 Stdout:63124905 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:12046847 Executing query SELECT count() FROM test_rename_parallel_same_node WHERE num2 % 1000 > 0 on node1 http://localhost:None "GET /v1.46/containers/4be7578f3f1d21e99da31eaf1ef6dc71af266c20fb8e73c115f8921f6d0834f0/json HTTP/1.1" 200 None Executing query INSERT INTO test_mutations FORMAT TSV on node5 Starting new HTTP connection (5): 172.16.7.8:9001 http://172.16.7.8:9001 "GET / HTTP/1.1" 200 0 Connected to Minio. http://172.16.7.8:9001 "GET /root?location= HTTP/1.1" 404 0 http://172.16.7.8:9001 "PUT /root HTTP/1.1" 200 0 S3 bucket 'root' created http://172.16.7.8:9001 "GET /root2?location= HTTP/1.1" 404 0 http://172.16.7.8:9001 "PUT /root2 HTTP/1.1" 200 0 S3 bucket 'root2' created ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --project-name roottestremovestalemovingparts-gw1 --file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/ch1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --project-name roottestremovestalemovingparts-gw1 --file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/ch1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml up -d --no-recreate] http://localhost:None "GET /v1.46/containers/4be7578f3f1d21e99da31eaf1ef6dc71af266c20fb8e73c115f8921f6d0834f0/json HTTP/1.1" 200 None ClickHouse node started Executing query CREATE TABLE test_table(date Date, id UInt32) ENGINE = ReplicatedMergeTree('/clickhouse/tables/shard1/thxARXx/test_table', '1') PARTITION BY toYYYYMM(date) ORDER BY id on node Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP TABLE IF EXISTS test_rename_parallel_same_node SYNC on node1 Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO test_table(date, id) select today(), number FROM numbers(1000) on node Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP TABLE IF EXISTS test_rename_parallel_same_node SYNC on node2 run container_id:roottestreloadzookeeper-gw5-node-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo \'\n\n \n \n zoo1\n 2181\n \n 2000\n \n\n\' > /etc/clickhouse-server/conf.d/zookeeper.xml'] Command:[docker exec roottestreloadzookeeper-gw5-node-1 bash -c echo ' zoo1 2181 2000 ' > /etc/clickhouse-server/conf.d/zookeeper.xml] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SYSTEM RELOAD CONFIG on node Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT COUNT() FROM test_table on node Executing query ALTER TABLE test_mutations UPDATE i = sleepEachRow(2) WHERE 1 on node5 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query DROP TABLE IF EXISTS test_rename_parallel_same_node SYNC on node3 Stdout:9 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:15252569 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Running Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Running Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Running Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Running Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Running Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Running Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Running Stderr: Container roottestremovestalemovingparts-gw1-ch1-1 Creating Stderr: Container roottestremovestalemovingparts-gw1-ch1-1 Created Stderr: Container roottestremovestalemovingparts-gw1-ch1-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-ch1-1 Started ClickHouse instance created get_instance_ip instance_name=ch1 http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-ch1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=ch1 http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-ch1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in ch1, ip: 172.16.7.9... http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-ch1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/6c1ec8372718febadcb73e85e1d83a0704f19ae68efc64b4eea283c025a995b2/json HTTP/1.1" 200 None Stopping zookeeper node: zoo1 Command:[docker compose --project-name roottestreloadzookeeper-gw5 --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop zoo1] Stdout:234626390 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query DROP TABLE IF EXISTS test_rename_parallel_same_node SYNC on node4 Stdout:63130119 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] http://localhost:None "GET /v1.46/containers/6c1ec8372718febadcb73e85e1d83a0704f19ae68efc64b4eea283c025a995b2/json HTTP/1.1" 200 None Stdout:15199648 http://localhost:None "GET /v1.46/containers/6c1ec8372718febadcb73e85e1d83a0704f19ae68efc64b4eea283c025a995b2/json HTTP/1.1" 200 None [gw3] PASSED test_rename_column/test.py::test_rename_parallel_same_node test_rename_column/test.py::test_rename_with_parallel_insert Executing query SELECT value FROM system.build_options WHERE name = 'CXX_FLAGS' on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query DROP TABLE IF EXISTS test_rename_with_parallel_insert SYNC on node1 http://localhost:None "GET /v1.46/containers/6c1ec8372718febadcb73e85e1d83a0704f19ae68efc64b4eea283c025a995b2/json HTTP/1.1" 200 None run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 1: insert for 2000-01-02: [9, 2, 7, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_insert SYNC on node2 No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 http://localhost:None "GET /v1.46/containers/6c1ec8372718febadcb73e85e1d83a0704f19ae68efc64b4eea283c025a995b2/json HTTP/1.1" 200 None http://localhost:None "POST /v1.46/exec/4bff5209355b876ae6650410f33c99bf37ad998ab5016a88bc2347c4bd790936/start HTTP/1.1" 200 0 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_insert SYNC on node3 http://localhost:None "GET /v1.46/exec/4bff5209355b876ae6650410f33c99bf37ad998ab5016a88bc2347c4bd790936/json HTTP/1.1" 200 586 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_insert SYNC on node4 http://localhost:None "GET /v1.46/containers/6c1ec8372718febadcb73e85e1d83a0704f19ae68efc64b4eea283c025a995b2/json HTTP/1.1" 200 None ClickHouse ch1 started Executing query CREATE DATABASE IF NOT EXISTS stale_moving_parts on ch1 Executing query CREATE TABLE test_rename_with_parallel_insert ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_insert', 'node1') ORDER BY num PARTITION BY num % 100 on node1 Executing query CREATE TABLE test_remove ON CLUSTER cluster ( id UInt32 ) ENGINE ReplicatedMergeTree() ORDER BY id; on ch1 Executing query CREATE TABLE test_rename_with_parallel_insert ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_insert', 'node2') ORDER BY num PARTITION BY num % 100 on node2 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 1: insert for 2000-01-02: [5, 5, 10, 10, 10, 5, 1, 8, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query CREATE TABLE test_rename_with_parallel_insert ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_insert', 'node3') ORDER BY num PARTITION BY num % 100 on node3 Stdout:21558869 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:234639128 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query SELECT data_paths FROM system.tables WHERE table = 'test_remove' and database = 'stale_moving_parts' LIMIT 1 on ch1 Executing query CREATE TABLE test_rename_with_parallel_insert ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_insert', 'node4') ORDER BY num PARTITION BY num % 100 on node4 Stdout:63144177 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query SYSTEM ENABLE FAILPOINT stop_moving_part_before_swap_with_active on ch1 Stdout:22556382 thread 0: insert for 2000-01-01: [6, 8, 10, 7, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO test_remove SELECT number FROM numbers(100); on ch1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(1000) on node1 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Stdout:9 Executing query ALTER TABLE test_remove MOVE PART 'all_0_0_0' TO DISK 's3' on ch1 Executing query SELECT part_name FROM system.moves on ch1 thread 1: insert for 2000-01-02: [10, 2, 10, 10, 4, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 thread 0: insert for 2000-01-01: [4, 1, 8, 3, 8, 7, 2, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:28917583 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:234659918 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:63163647 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 1: insert for 2000-01-02: [7, 4, 4, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:6071 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3'settings keeper_max_retries=3, keeper_retry_max_backoff_ms=500 on node1 Stdout:32017020 Stdout:6071 Executing query select 20 on node1 thread 0: insert for 2000-01-01: [3, 10, 8, 4, 4, 3, 8, 5, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo2 to foo3 on node2 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo3 to num2 on node3 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(100) on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(100) on node2 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(100) on node3 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo2 to foo3 on node2 http://localhost:None "POST /v1.46/containers/322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/d47e6784f572d13925afea28183ad3f1eb29ee8416d7156a9e8f42658f0119df/start HTTP/1.1" 101 0 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(100) on node1 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo3 to num2 on node3 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(100) on node3 http://localhost:None "GET /v1.46/exec/d47e6784f572d13925afea28183ad3f1eb29ee8416d7156a9e8f42658f0119df/json HTTP/1.1" 200 544 [network] 322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62: ['iptables', '--wait', '-D', 'DOCKER-USER', '-m', 'statistic', '--mode', 'random', '--probability', '0.01', '-p', 'tcp', '-s', '172.16.9.9', '--dport', '2181', '-j', 'REJECT', '--reject-with', 'tcp-reset'] (0): http://localhost:None "POST /v1.46/containers/322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/595442fa36730309c4ddf68e924ea2854193808e52c01827bd223c5a5aa018ce/start HTTP/1.1" 101 0 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(100) on node1 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo3 to num2 on node3 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo3 to num2 on node3 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(100) on node1 thread 1: insert for 2000-01-02: [3, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 http://localhost:None "GET /v1.46/exec/595442fa36730309c4ddf68e924ea2854193808e52c01827bd223c5a5aa018ce/json HTTP/1.1" 200 544 [network] 322e2bc5b94f91043fe495ea4378568646ddac5e6ef06cee79b6dd5e69d9bf62: ['iptables', '--wait', '-D', 'DOCKER-USER', '-m', 'statistic', '--mode', 'random', '--probability', '0.01', '-p', 'tcp', '-d', '172.16.9.9', '--sport', '2181', '-j', 'REJECT', '--reject-with', 'tcp-reset'] (0): run container_id:roottestreplicatedmergetrees3zerocopy-gw8-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'timeout 60 tail -Fn1000000 "/var/log/clickhouse-server/clickhouse-server.log" | grep -Em 1 \'Removing metadata metadata_dropped/default.test_drop_table\''] Command:[docker exec roottestreplicatedmergetrees3zerocopy-gw8-node1-1 bash -c timeout 60 tail -Fn1000000 "/var/log/clickhouse-server/clickhouse-server.log" | grep -Em 1 'Removing metadata metadata_dropped/default.test_drop_table'] Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo3 to num2 on node3 thread 0: insert for 2000-01-01: [2, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,foo3) SELECT number + 0 AS num, number + 1 + 0 AS foo3 FROM numbers_mt(100) on node1 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:9 Stdout:33120309 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SELECT part_name FROM system.moves on ch1 Stdout:234662690 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestremovestalemovingparts-gw1-ch1-1 detach:False nothrow:False cmd: ['bash', '-c', 'ls /var/lib/clickhouse/disks/s3/store/8af/8aff6f37-927b-406a-9a36-fc430f60964d/moving'] Command:[docker exec roottestremovestalemovingparts-gw1-ch1-1 bash -c ls /var/lib/clickhouse/disks/s3/store/8af/8aff6f37-927b-406a-9a36-fc430f60964d/moving] Stdout:63166419 run container_id:roottestreplicatedfetchesbandwidth-gw6-node3-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node3-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:all_0_0_0 Stopping zookeeper node: zoo1 Command:[docker compose --project-name roottestremovestalemovingparts-gw1 --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop zoo1] Stdout:33067388 thread 1: insert for 2000-01-02: [8, 1, 2, 6, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [5, 2, 3, 1, 4, 2, 1, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(100) on node2 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(100) on node3 Executing query DROP TABLE IF EXISTS limited_sends0 SYNC on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(100) on node2 thread 1: insert for 2000-01-02: [7, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(100) on node2 grep in log called Reloaded certificate (/etc/clickhouse-server/config.d/third_client.crt) run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -a "Reloaded certificate (/etc/clickhouse-server/config.d/third_client.crt)" /var/log/clickhouse-server/clickhouse-server.log* || true'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -a "Reloaded certificate (/etc/clickhouse-server/config.d/third_client.crt)" /var/log/clickhouse-server/clickhouse-server.log* || true] thread 0: insert for 2000-01-01: [1, 5, 7, 6, 4, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(100) on node2 Stdout:2025.04.02 03:32:12.138801 [ 670 ] {} CertificateReloader: Reloaded certificate (/etc/clickhouse-server/config.d/third_client.crt) and key (/etc/clickhouse-server/config.d/third_client.key). grep result 2025.04.02 03:32:12.138801 [ 670 ] {} CertificateReloader: Reloaded certificate (/etc/clickhouse-server/config.d/third_client.crt) and key (/etc/clickhouse-server/config.d/third_client.key). Stopping zookeeper node: zoo1 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo1] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(100) on node3 thread 1: insert for 2000-01-02: [2, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [8, 6, 2, 6, 3, 9, 10, 8, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestpostgresqlreplicadatabaseengine1-gw0-instance-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/cb3b19b3d0f83c8e3adb8c80699596e70da5c2f8c04a409bdca12a91ee98956f/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/cb3b19b3d0f83c8e3adb8c80699596e70da5c2f8c04a409bdca12a91ee98956f/json HTTP/1.1" 200 586 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN num2 to foo2 on node1 thread 0: insert for 2000-01-01: [2, 10, 9, 9, 7, 7, 5, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN num2 to foo2 on node1 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN num2 to foo2 on node1 Stderr: Container roottestreloadmaxtablesizetodrop-gw7-node-1 Stopping Stderr: Container roottestreloadmaxtablesizetodrop-gw7-node-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/.env --project-name roottestreloadmaxtablesizetodrop-gw7 --file /ClickHouse/tests/integration/test_reload_max_table_size_to_drop/_instances-0-gw7/node/docker-compose.yml down --volumes] thread 1: insert for 2000-01-02: [7, 2, 10, 5, 10, 9, 9, 6, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN num2 to foo2 on node1 Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopped Stopping zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo2] Executing query CREATE TABLE test_remove_disk_from_policy ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 Stdout:2025.04.02 03:32:22.247218 [ 678 ] {} DatabaseCatalog: Removing metadata metadata_dropped/default.test_drop_table.6a8bcc64-2c90-4284-82b2-aadfca9c8d1e.sql of dropped table default.test_drop_table (6a8bcc64-2c90-4284-82b2-aadfca9c8d1e) 1 log line(s) matching "Removing metadata metadata_dropped/default.test_drop_table" appeared in a 1.812 seconds Executing query select name from system.zookeeper where path='/test/drop_table/replicas' on node1 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_insert (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(100) on node3 thread 0: insert for 2000-01-01: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT volume_name FROM system.storage_policies WHERE policy_name = 'test_remove_disk_from_policy_cool_policy' on node1 Executing query create table if not exists test_drop_table (n int) engine=ReplicatedMergeTree('/test/drop_table', '1') order by n partition by n % 99 settings storage_policy='s3' on node1 thread 1: insert for 2000-01-02: [1, 9, 10, 2, 6, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT disks FROM system.storage_policies WHERE policy_name = 'test_remove_disk_from_policy_cool_policy' on node1 Executing query SYSTEM RELOAD CONFIG on node1 Executing query SELECT volume_name FROM system.storage_policies WHERE policy_name = 'test_remove_disk_from_policy_cool_policy' on node1 Executing query detach table test_drop_table sync on node1 Executing query DROP TABLE IF EXISTS limited_sends1 SYNC on node1 Executing query SELECT disks FROM system.storage_policies WHERE policy_name = 'test_remove_disk_from_policy_cool_policy' on node1 Stderr: Container roottestreloadmaxtablesizetodrop-gw7-node-1 Stopping Stderr: Container roottestreloadmaxtablesizetodrop-gw7-node-1 Stopped Stderr: Container roottestreloadmaxtablesizetodrop-gw7-node-1 Removing Stderr: Container roottestreloadmaxtablesizetodrop-gw7-node-1 Removed Stderr: Network roottestreloadmaxtablesizetodrop-gw7_default Removing Stderr: Network roottestreloadmaxtablesizetodrop-gw7_default Removed Cleanup called thread 0: insert for 2000-01-01: [4, 2, 5, 2, 4, 4, 3, 6, 3, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Docker networks for project roottestreloadmaxtablesizetodrop-gw7 are NETWORK ID NAME DRIVER SCOPE Executing query attach table test_drop_table on node1 Docker containers for project roottestreloadmaxtablesizetodrop-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreloadmaxtablesizetodrop-gw7 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreloadmaxtablesizetodrop-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreloadmaxtablesizetodrop-gw7 Trying to prune unused networks... run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Trying to prune unused images... Command:[docker image prune -f] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c cat /var/log/clickhouse-server/clickhouse-server.log] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:9 Command:[docker volume prune -f] Stdout:795 Clickhouse process running. run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:Total reclaimed space: 0B Volumes pruned: 9 Running tests in /ClickHouse/tests/integration/test_reload_query_masking_rules/test.py Cluster start called. is_up=False Stdout:2025.04.02 03:32:19.010810 [ 6071 ] {} SentryWriter: Sending crash reports is disabled test_reload_query_masking_rules/test.py::test_reload_query_masking_rules Stdout:2025.04.02 03:32:19.055923 [ 6071 ] {} Application: Starting ClickHouse 25.2.2.96963.altinityantalya.96963 (revision: 54496, git hash: 25f7c2a766916d92d37aee82d9defa5f14b86ec8, build id: 4783FC52DD13D5DFB0294BDDD711047195FEB5A6), PID 6071 Stdout:2025.04.02 03:32:19.056062 [ 6071 ] {} Application: starting up Stdout:2025.04.02 03:32:19.056072 [ 6071 ] {} Application: OS name: Linux, version: 5.15.0-130-generic, architecture: x86_64 Stdout:2025.04.02 03:32:19.056185 [ 6071 ] {} Jemalloc: Value for background_thread set to true (from true) Stdout:2025.04.02 03:32:19.060015 [ 6071 ] {} Application: Available RAM: 30.60 GiB; logical cores: 16; used cores: 16. Stdout:2025.04.02 03:32:19.060048 [ 6071 ] {} Application: Available CPU instruction sets: SSE, SSE2, SSE3, SSSE3, SSE41, SSE42, F16C, POPCNT, BMI1, BMI2, PCLMUL, AES, AVX, FMA, AVX2, SHA, ADX, RDRAND, RDSEED, RDTSCP, CLFLUSHOPT, CLWB, XSAVE, OSXSAVE Stdout:2025.04.02 03:32:19.060075 [ 6071 ] {} Pipe: Pipe capacity is 1.00 MiB Stdout:2025.04.02 03:32:19.060915 [ 6071 ] {} CgroupsReader: Will create cgroup reader from '/sys/fs/cgroup/' (cgroups version: v2) Stdout:2025.04.02 03:32:19.061109 [ 6071 ] {} AsynchronousMetrics: Scanning /sys/class/thermal Stdout:2025.04.02 03:32:19.061127 [ 6071 ] {} AsynchronousMetrics: Scanning /sys/block Stdout:2025.04.02 03:32:19.061205 [ 6071 ] {} AsynchronousMetrics: Scanning /sys/devices/system/edac Stdout:2025.04.02 03:32:19.061220 [ 6071 ] {} AsynchronousMetrics: Scanning /sys/class/hwmon Stdout:2025.04.02 03:32:19.061295 [ 6071 ] {} StatusFile: Status file /var/lib/clickhouse/status already exists - unclean restart. Contents: Stdout:PID: 5299 Stdout:Started at: 2025-04-02 03:32:14 Stdout:Revision: 54496 Stdout: Stdout:2025.04.02 03:32:19.207697 [ 6071 ] {} Application: Integrity check of the executable successfully passed (checksum: 2D3F861318CFEDFBBB742B17F83CC397) Stdout:2025.04.02 03:32:19.207796 [ 6071 ] {} Application: Will do mlock to prevent executable memory from being paged out. It may take a few seconds. Stdout:2025.04.02 03:32:19.211593 [ 6071 ] {} Application: The memory map of clickhouse executable has been mlock'ed, total 287.17 MiB Stdout:2025.04.02 03:32:19.211625 [ 6071 ] {} MemoryWorker: Starting background memory thread with period of 50ms, using Cgroups as source Stdout:2025.04.02 03:32:19.211717 [ 6071 ] {} BackgroundSchedulePool/BgSchPool: Create BackgroundSchedulePool with 512 threads Stdout:2025.04.02 03:32:19.263485 [ 6087 ] {} MemoryTracker: Correcting the value of global memory tracker from 2.57 MiB to 93.26 MiB Stdout:2025.04.02 03:32:19.280156 [ 6071 ] {} Application: rlimit on number of file descriptors is 1048576 Stdout:2025.04.02 03:32:19.280182 [ 6071 ] {} Application: rlimit on number of threads is 18446744073709551615 Stdout:2025.04.02 03:32:19.280196 [ 6071 ] {} Application: Initializing DateLUT. Stdout:2025.04.02 03:32:19.280201 [ 6071 ] {} Application: Initialized DateLUT with time zone 'Etc/UTC'. Stdout:2025.04.02 03:32:19.280158 [ 6088 ] {} CancellationChecker: Started worker function Stdout:2025.04.02 03:32:19.280236 [ 6071 ] {} Context: Setting up /var/lib/clickhouse/tmp/ to store temporary data in it Stdout:2025.04.02 03:32:19.280442 [ 6071 ] {} Application: Configuration parameter 'interserver_http_host' doesn't exist or exists and empty. Will use 'node1' as replica host. Stdout:2025.04.02 03:32:19.280457 [ 6071 ] {} Application: Initializing interserver credentials. Stdout:2025.04.02 03:32:19.280565 [ 6071 ] {} NamedCollectionsMetadataStorage: Using local storage for named collections at path: /var/lib/clickhouse/named_collections Stdout:2025.04.02 03:32:19.280586 [ 6071 ] {} FileCacheFactory: Will load 0 caches from default cache config Stdout:2025.04.02 03:32:19.282173 [ 6071 ] {} ConfigReloader: Loading config '/etc/clickhouse-server/config.xml' Stdout:2025.04.02 03:32:19.282183 [ 6071 ] {} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/config.xml'. Stdout:2025.04.02 03:32:19.282986 [ 6071 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:19.283073 [ 6071 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:19.283158 [ 6071 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_enable_keeper_async_replication.xml'. Stdout:2025.04.02 03:32:19.283212 [ 6071 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_instance_config.xml'. Stdout:2025.04.02 03:32:19.283360 [ 6071 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/logs_config.xml'. Stdout:2025.04.02 03:32:19.283442 [ 6071 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/storage_configuration.xml'. Stdout:2025.04.02 03:32:19.287656 [ 6071 ] {} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'. Stdout:2025.04.02 03:32:19.287749 [ 6071 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performing update on configuration Stdout:2025.04.02 03:32:19.303372 [ 6071 ] {} Application: Changed setting 'max_server_memory_usage' to 27.54 GiB (30.60 GiB available memory * 0.90 max_server_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:19.303402 [ 6071 ] {} Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.30 GiB (30.60 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:19.303411 [ 6071 ] {} Application: Merges and mutations memory limit is set to 15.30 GiB Stdout:2025.04.02 03:32:19.303684 [ 6071 ] {} Application: Setting max_remote_read_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:19.303708 [ 6071 ] {} Application: Setting max_remote_write_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:19.303721 [ 6071 ] {} Application: ConcurrencyControl limit is set to 32 CPU slots with 'fair_round_robin' scheduler Stdout:2025.04.02 03:32:19.303740 [ 6071 ] {} BackgroundSchedulePool/BgBufSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:19.305547 [ 6071 ] {} BackgroundSchedulePool/BgMBSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:19.307986 [ 6071 ] {} BackgroundSchedulePool/BgDistSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:19.310342 [ 6071 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration Docker networks for project roottestreloadquerymaskingrules-gw7 are NETWORK ID NAME DRIVER SCOPE Stdout:2025.04.02 03:32:19.310371 [ 6071 ] {} ConfigReloader: Config reload interval set to 2000ms Stdout:2025.04.02 03:32:19.310798 [ 6071 ] {} Application: Listening for replica communication (interserver): http://0.0.0.0:9009 Stdout:2025.04.02 03:32:19.312592 [ 6071 ] {} ConfigReloader: Loading config '/etc/clickhouse-server/users.xml' Stdout:2025.04.02 03:32:19.312601 [ 6071 ] {} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/users.xml'. Stdout:2025.04.02 03:32:19.312755 [ 6071 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:19.312794 [ 6071 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:19.312836 [ 6071 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/users.d/0_common_instance_users.xml'. Stdout:2025.04.02 03:32:19.312865 [ 6071 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/users.d/0_random_settings.xml'. Stdout:2025.04.02 03:32:19.313413 [ 6071 ] {} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'. Stdout:2025.04.02 03:32:19.313432 [ 6071 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performing update on configuration Stdout:2025.04.02 03:32:19.313931 [ 6071 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performed update on configuration Stdout:2025.04.02 03:32:19.313943 [ 6071 ] {} ConfigReloader: Config reload interval set to 2000ms Stdout:2025.04.02 03:32:19.314129 [ 6071 ] {} Access(user directories): Added users_xml access storage 'users_xml', path: /etc/clickhouse-server/users.xml Stdout:2025.04.02 03:32:19.314224 [ 6071 ] {} Access(user directories): Added local_directory access storage 'local_directory', path: /var/lib/clickhouse/access/ Stdout:2025.04.02 03:32:19.314300 [ 6071 ] {} CgroupsMemoryUsageObserver: Started cgroup current memory usage observer thread Stdout:2025.04.02 03:32:19.314604 [ 6654 ] {} CgroupsMemoryUsageObserver: Memory amount initially available to the process is 30.60 GiB Stdout:2025.04.02 03:32:19.316852 [ 6071 ] {} Context: Initialized background executor for merges and mutations with num_threads=16, num_tasks=32, scheduling_policy=round_robin Stdout:2025.04.02 03:32:19.317766 [ 6071 ] {} Context: Initialized background executor for move operations with num_threads=8, num_tasks=8 Stdout:2025.04.02 03:32:19.319587 [ 6071 ] {} Context: Initialized background executor for fetches with num_threads=16, num_tasks=16 Stdout:2025.04.02 03:32:19.320416 [ 6071 ] {} Context: Initialized background executor for common operations (e.g. clearing old parts) with num_threads=8, num_tasks=8 Stdout:2025.04.02 03:32:19.322705 [ 6071 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. Stdout:2025.04.02 03:32:19.323465 [ 6071 ] {} DNSCacheUpdater: Update period 15 seconds Stdout:2025.04.02 03:32:19.323500 [ 6071 ] {} Application: Loading metadata from /var/lib/clickhouse/ Stdout:2025.04.02 03:32:19.323541 [ 6089 ] {} DNSResolver: Updating DNS cache Stdout:2025.04.02 03:32:19.323588 [ 6089 ] {} DNSResolver: Updated DNS cache Stdout:2025.04.02 03:32:19.325581 [ 6071 ] {} Context: Database disk name: default Stdout:2025.04.02 03:32:19.325599 [ 6071 ] {} Context: Database disk name: default, path: /var/lib/clickhouse/ Stdout:2025.04.02 03:32:19.325626 [ 6071 ] {} loadSystemDatabase: metadata_file_path metadata/system.sql, existsFile true Stdout:2025.04.02 03:32:19.325708 [ 6071 ] {} loadSystemDatabase: metadata_file_path metadata/information_schema.sql, existsFile true Stdout:2025.04.02 03:32:19.325747 [ 6071 ] {} loadSystemDatabase: metadata_file_path metadata/INFORMATION_SCHEMA.sql, existsFile true Stdout:2025.04.02 03:32:19.325826 [ 6071 ] {} DatabaseAtomic (system): Metadata processed, database system has 0 tables and 0 dictionaries in total. Stdout:2025.04.02 03:32:19.325835 [ 6071 ] {} TablesLoader: Parsed metadata of 0 tables in 1 databases in 5.878e-05 sec Stdout:2025.04.02 03:32:19.325839 [ 6071 ] {} ReferentialDeps: No tables Stdout:2025.04.02 03:32:19.325844 [ 6071 ] {} LoadingDeps: No tables Stdout:2025.04.02 03:32:19.325882 [ 6071 ] {} AsyncLoader: Prioritize load job 'startup Atomic database system': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:19.325886 [ 6071 ] {} AsyncLoader: Prioritize load job 'startup Ordinary database system': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:19.325893 [ 6071 ] {} AsyncLoader: Schedule load job 'startup Ordinary database system' into ForegroundLoad Stdout:2025.04.02 03:32:19.325896 [ 6071 ] {} AsyncLoader: Schedule load job 'startup Atomic database system' into ForegroundLoad Stdout:2025.04.02 03:32:19.325900 [ 6071 ] {} AsyncLoader: Change current priority: none -> 0 Stdout:2025.04.02 03:32:19.325904 [ 6071 ] {} AsyncLoader: Spawn loader worker #1 in ForegroundLoad Stdout:2025.04.02 03:32:19.325964 [ 6071 ] {} AsyncLoader: Wait load job 'startup Atomic database system' in ForegroundLoad Stdout:2025.04.02 03:32:19.326068 [ 6721 ] {} AsyncLoader: Execute load job 'startup Ordinary database system' in ForegroundLoad Stdout:2025.04.02 03:32:19.326114 [ 6721 ] {} AsyncLoader: Finish load job 'startup Ordinary database system' with status OK Stdout:2025.04.02 03:32:19.326129 [ 6721 ] {} AsyncLoader: Spawn loader worker #2 in ForegroundLoad Stdout:2025.04.02 03:32:19.326211 [ 6721 ] {} AsyncLoader: Execute load job 'startup Atomic database system' in ForegroundLoad Stdout:2025.04.02 03:32:19.326235 [ 6721 ] {} AsyncLoader: Finish load job 'startup Atomic database system' with status OK Stdout:2025.04.02 03:32:19.326276 [ 6721 ] {} AsyncLoader: Stop worker in ForegroundLoad Stdout:2025.04.02 03:32:19.326277 [ 6071 ] {} SystemLog: Creating system.query_log from query_log Stdout:2025.04.02 03:32:19.326318 [ 6722 ] {} AsyncLoader: Stop worker in ForegroundLoad Stdout:2025.04.02 03:32:19.326350 [ 6722 ] {} AsyncLoader: Change current priority: 0 -> none Stdout:2025.04.02 03:32:19.334967 [ 6071 ] {} SystemLog: Creating system.query_thread_log from query_thread_log Stdout:2025.04.02 03:32:19.335431 [ 6071 ] {} SystemLog: Creating system.part_log from part_log Stdout:2025.04.02 03:32:19.335752 [ 6071 ] {} SystemLog: Creating system.trace_log from trace_log Stdout:2025.04.02 03:32:19.336021 [ 6071 ] {} SystemLog: Creating system.crash_log from crash_log Stdout:2025.04.02 03:32:19.336181 [ 6071 ] {} SystemLog: Creating system.text_log from text_log Stdout:2025.04.02 03:32:19.336401 [ 6071 ] {} SystemLog: Creating system.metric_log from metric_log Stdout:2025.04.02 03:32:19.340772 [ 6071 ] {} SystemLog: Creating system.latency_log from latency_log Stdout:2025.04.02 03:32:19.341024 [ 6071 ] {} SystemLog: Creating system.error_log from error_log Stdout:2025.04.02 03:32:19.341241 [ 6071 ] {} SystemLog: Not creating system.filesystem_cache_log since corresponding section 'filesystem_cache_log' is missing from config Stdout:2025.04.02 03:32:19.341256 [ 6071 ] {} SystemLog: Not creating system.filesystem_read_prefetches_log since corresponding section 'filesystem_read_prefetches_log' is missing from config Stdout:2025.04.02 03:32:19.341266 [ 6071 ] {} SystemLog: Creating system.s3queue_log from s3queue_log Stdout:2025.04.02 03:32:19.341444 [ 6071 ] {} SystemLog: Not creating system.azure_queue_log since corresponding section 'azure_queue_log' is missing from config Stdout:2025.04.02 03:32:19.341457 [ 6071 ] {} SystemLog: Creating system.asynchronous_metric_log from asynchronous_metric_log Stdout:2025.04.02 03:32:19.341641 [ 6071 ] {} SystemLog: Creating system.opentelemetry_span_log from opentelemetry_span_log Stdout:2025.04.02 03:32:19.341870 [ 6071 ] {} SystemLog: Creating system.query_views_log from query_views_log Stdout:2025.04.02 03:32:19.342148 [ 6071 ] {} SystemLog: Not creating system.zookeeper_log since corresponding section 'zookeeper_log' is missing from config Stdout:2025.04.02 03:32:19.342163 [ 6071 ] {} SystemLog: Not creating system.session_log since corresponding section 'session_log' is missing from config Stdout:2025.04.02 03:32:19.342172 [ 6071 ] {} SystemLog: Not creating system.transactions_info_log since corresponding section 'transactions_info_log' is missing from config Stdout:2025.04.02 03:32:19.342181 [ 6071 ] {} SystemLog: Creating system.processors_profile_log from processors_profile_log Stdout:2025.04.02 03:32:19.342474 [ 6071 ] {} SystemLog: Creating system.asynchronous_insert_log from asynchronous_insert_log Stdout:2025.04.02 03:32:19.342757 [ 6071 ] {} SystemLog: Creating system.backup_log from backup_log Stdout:2025.04.02 03:32:19.343042 [ 6071 ] {} SystemLog: Creating system.blob_storage_log from blob_storage_log Stdout:2025.04.02 03:32:19.343385 [ 6071 ] {} SystemLog: Creating system.query_metric_log from query_metric_log Stdout:2025.04.02 03:32:19.347960 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.347979 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.349683 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.349700 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.349720 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.349724 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.349739 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.349742 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.349756 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.349759 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.349803 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.349808 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.349896 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.349902 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.349941 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.349944 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.349985 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.349989 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350041 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350048 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350082 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350086 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350119 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350123 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350152 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350155 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350185 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350189 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350219 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350225 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350260 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350264 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350285 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350288 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350307 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350310 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350334 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350337 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350353 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350358 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350376 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350379 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350397 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350400 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350416 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350419 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350444 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350447 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350465 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Docker containers for project roottestreloadquerymaskingrules-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Stdout:2025.04.02 03:32:19.350470 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopping Stdout:2025.04.02 03:32:19.350519 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopped Stdout:2025.04.02 03:32:19.350522 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stopping zookeeper node: zoo3 Stdout:795 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo3] Stdout:2025.04.02 03:32:19.350545 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350548 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Executing query select 20 on instance Stdout:2025.04.02 03:32:19.350673 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350679 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350705 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350708 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350726 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350732 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350751 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350754 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350779 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350783 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350823 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350827 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350859 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350863 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350902 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350913 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.350957 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.350961 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351021 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351039 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351111 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351115 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351147 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351150 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351235 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351239 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351263 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351267 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351283 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351286 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351307 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351310 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351339 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351343 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351364 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351368 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351383 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351386 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351424 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351428 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351457 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351461 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351483 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351486 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351625 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351630 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351709 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351718 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351751 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351754 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351773 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351776 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351813 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351816 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351874 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351879 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351899 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351902 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.351917 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.351920 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352050 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352056 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352178 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352182 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352243 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352248 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352342 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352346 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352438 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352442 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352487 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352490 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352523 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352526 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352686 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352694 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352742 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352746 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352829 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352835 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352877 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352881 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.352917 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.352920 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353058 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353064 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353117 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353120 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353172 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353176 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353207 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353210 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353261 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353268 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353298 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353302 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353454 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353461 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353498 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353502 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353520 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353523 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353550 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353556 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353588 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353592 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353620 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353624 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353654 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353658 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353710 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353714 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353744 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353755 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353782 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353785 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353818 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353821 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353846 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353849 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353895 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.353899 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.353999 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354008 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354045 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354049 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354077 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354081 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354105 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354108 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354131 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354134 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354155 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354168 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354186 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354190 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354227 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354230 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354253 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354257 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354277 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354280 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354312 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354318 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354356 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354360 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.354435 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.354439 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.361504 [ 6071 ] {} DatabaseCatalog: Found 7 partially dropped tables. Will load them and retry removal. Stdout:2025.04.02 03:32:19.361900 [ 6741 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3) from metadata_dropped/default.test_add_disk.8a487b06-210e-4c1c-91cf-235fb0fcffb3.sql Stdout:2025.04.02 03:32:19.362292 [ 6743 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174) from metadata_dropped/default.test_add_disk.ab1b9e7f-3e87-42e0-987c-542af287b174.sql Stdout:2025.04.02 03:32:19.362947 [ 6746 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c) from metadata_dropped/default.test_add_policy.d69b6b75-0af1-4149-a0f9-c13bb05ea67c.sql Stdout:2025.04.02 03:32:19.363066 [ 6741 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:19.363160 [ 6747 ] {} DatabaseCatalog: Trying load partially dropped table default.test_new_policy_works (9288e96b-23dc-4115-80b9-90a084e01018) from metadata_dropped/default.test_new_policy_works.9288e96b-23dc-4115-80b9-90a084e01018.sql Stdout:2025.04.02 03:32:19.364161 [ 6747 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:19.364198 [ 6744 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f) from metadata_dropped/default.test_add_disk_to_policy.e1af3e60-fe6e-4895-a02b-fced2189743f.sql Stdout:2025.04.02 03:32:19.364372 [ 6741 ] {} StoragePolicy (small_jbod_with_external): Storage policy small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:19.364398 [ 6741 ] {} StoragePolicySelector: Storage policy `small_jbod_with_external` loaded Stdout:2025.04.02 03:32:19.364320 [ 6745 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5) from metadata_dropped/default.test_add_volume_to_policy.b03aef8f-055b-454d-bac3-7055c6fd12b5.sql Stdout:2025.04.02 03:32:19.364502 [ 6744 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:19.364550 [ 6741 ] {} StoragePolicy (one_more_small_jbod_with_external): Storage policy one_more_small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:19.364556 [ 6741 ] {} StoragePolicySelector: Storage policy `one_more_small_jbod_with_external` loaded Stdout:2025.04.02 03:32:19.364590 [ 6743 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:19.364656 [ 6746 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:19.364742 [ 6741 ] {} StoragePolicy (jbods_with_external): Storage policy jbods_with_external created, total volumes 2 Executing query system sync replica test_drop_table on node1 Stdout:2025.04.02 03:32:19.364750 [ 6741 ] {} StoragePolicySelector: Storage policy `jbods_with_external` loaded Stdout:2025.04.02 03:32:19.364942 [ 6741 ] {} StoragePolicy (moving_jbod_with_external): Storage policy moving_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:19.364957 [ 6741 ] {} StoragePolicySelector: Storage policy `moving_jbod_with_external` loaded Stdout:2025.04.02 03:32:19.365333 [ 6745 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:19.365374 [ 6749 ] {} DatabaseCatalog: Trying load partially dropped table default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce) from metadata_dropped/default.test_remove_disk.b18b3533-cce9-430c-ba4b-cbd718231cce.sql Stdout:2025.04.02 03:32:19.365451 [ 6741 ] {} StorageConfiguration: Volume `small` max_data_part_size is too low (2.00 MiB < 8.00 MiB) Stdout:2025.04.02 03:32:19.365563 [ 6741 ] {} StoragePolicy (default_disk_with_external): Storage policy default_disk_with_external created, total volumes 2 Stdout:2025.04.02 03:32:19.365575 [ 6741 ] {} StoragePolicySelector: Storage policy `default_disk_with_external` loaded Stdout:2025.04.02 03:32:19.365666 [ 6749 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:19.365780 [ 6741 ] {} StorageConfiguration: Volume `special_warning_small_volume` max_data_part_size is too low (1.00 KiB < 8.00 MiB) Stdout:2025.04.02 03:32:19.365899 [ 6741 ] {} StoragePolicy (special_warning_policy): Storage policy special_warning_policy created, total volumes 4 Stdout:2025.04.02 03:32:19.365905 [ 6741 ] {} StoragePolicySelector: Storage policy `special_warning_policy` loaded Stdout:2025.04.02 03:32:19.366040 [ 6741 ] {} StoragePolicy (test_remove_disk_from_policy_cool_policy): Storage policy test_remove_disk_from_policy_cool_policy created, total volumes 1 Stdout:2025.04.02 03:32:19.366048 [ 6741 ] {} StoragePolicySelector: Storage policy `test_remove_disk_from_policy_cool_policy` loaded Stdout:2025.04.02 03:32:19.366362 [ 6741 ] {} StoragePolicy (default): Storage policy default created, total volumes 1 Stdout:2025.04.02 03:32:19.366712 [ 6741 ] {} default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3): Loading data parts Stdout:2025.04.02 03:32:19.366836 [ 6743 ] {} default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174): Loading data parts Stdout:2025.04.02 03:32:19.367073 [ 6746 ] {} default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c): Loading data parts Stdout:2025.04.02 03:32:19.367110 [ 6745 ] {} default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5): Loading data parts Stdout:2025.04.02 03:32:19.367807 [ 6744 ] {} default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f): Loading data parts Stdout:2025.04.02 03:32:19.367952 [ 6749 ] {} default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce): Loading data parts Stdout:2025.04.02 03:32:19.368987 [ 6741 ] {} default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3): There are no data parts Stdout:2025.04.02 03:32:19.370246 [ 6747 ] {} DatabaseCatalog: Cannot load partially dropped table default.test_new_policy_works (9288e96b-23dc-4115-80b9-90a084e01018) from: metadata_dropped/default.test_new_policy_works.9288e96b-23dc-4115-80b9-90a084e01018.sql. Parsed query: ATTACH TABLE default.test_new_policy_works UUID '9288e96b-23dc-4115-80b9-90a084e01018' (`d` UInt64) ENGINE = MergeTree ORDER BY d SETTINGS storage_policy = 'cool_policy', index_granularity = 8192. Will remove metadata and store/928/9288e96b-23dc-4115-80b9-90a084e01018/. Garbage may be left in ZooKeeper.: Code: 478. DB::Exception: Unknown storage policy `cool_policy`. (UNKNOWN_POLICY), Stack trace (when copying this message, always include the lines below): Stdout: Stdout:0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000cfb8fbb Stdout:1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007ff830c Stdout:2. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x0000000007ff802b Stdout:3. DB::StoragePolicySelector::get(String const&) const @ 0x00000000105c18e1 Stdout:4. DB::Context::getStoragePolicy(String const&) const @ 0x0000000010b34cbf Stdout:5. DB::MergeTreeData::getStoragePolicy() const @ 0x000000001201df88 Stdout:6. DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x00000000120136a8 Stdout:7. DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x00000000123e6548 Stdout:8. DB::create(DB::StorageFactory::Arguments const&) @ 0x00000000123e2c8c Stdout:9. DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x00000000119e37bf Stdout:10. DB::createTableFromAST(DB::ASTCreateQuery, String const&, String const&, std::shared_ptr, DB::LoadingStrictnessLevel) @ 0x00000000104454aa Stdout:11. DB::DatabaseCatalog::enqueueDroppedTableCleanup(DB::StorageID, std::shared_ptr, String, bool) @ 0x0000000010c038a9 Stdout:12. void std::__function::__policy_invoker::__call_impl[abi:ne190107]>(std::__function::__policy_storage const*) @ 0x0000000010c14b0e Stdout:13. void DB::ThreadPoolCallbackRunnerLocal>, std::function>::executeCallback>(std::promise&, std::function&&) @ 0x000000000efb3e95 Stdout:14. DB::ThreadPoolCallbackRunnerLocal>, std::function>::operator()(std::function&&, Priority, std::optional)::'lambda'()::operator()() @ 0x000000000efb3c9b Stdout:15. ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x000000000d0992db Stdout:16. void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl>::ThreadFromThreadPool::*)(), ThreadPoolImpl>::ThreadFromThreadPool*>(void (ThreadPoolImpl>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl>::ThreadFromThreadPool*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000d09fda3 Stdout:17. ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000000d09656f Stdout:18. void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000000d09d85a Stdout:19. ? @ 0x00007fb6b9fe6ac3 Stdout:20. ? @ 0x00007fb6ba078850 Stdout: (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:19.370496 [ 6743 ] {} default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174): There are no data parts Stdout:2025.04.02 03:32:19.371276 [ 6746 ] {} default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c): There are no data parts Stdout:2025.04.02 03:32:19.371677 [ 6744 ] {} default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f): There are no data parts Stdout:2025.04.02 03:32:19.378303 [ 6745 ] {} default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5): There are no data parts Stdout:2025.04.02 03:32:19.378361 [ 6749 ] {} default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce): There are no data parts Stdout:2025.04.02 03:32:19.378676 [ 6071 ] {} DatabaseAtomic (default): Metadata processed, database default has 0 tables and 0 dictionaries in total. Stdout:2025.04.02 03:32:19.378688 [ 6071 ] {} TablesLoader: Parsed metadata of 0 tables in 1 databases in 4.1567e-05 sec Stdout:2025.04.02 03:32:19.378694 [ 6071 ] {} ReferentialDeps: No tables Stdout:2025.04.02 03:32:19.378697 [ 6071 ] {} LoadingDeps: No tables Stdout:2025.04.02 03:32:19.378723 [ 6071 ] {} loadMetadata: Start asynchronous loading of databases Stdout:2025.04.02 03:32:19.378732 [ 6071 ] {} AsyncLoader: Schedule load job 'startup Ordinary database default' into BackgrndStartup Stdout:2025.04.02 03:32:19.378738 [ 6071 ] {} AsyncLoader: Schedule load job 'startup Atomic database default' into BackgrndStartup Stdout:2025.04.02 03:32:19.378745 [ 6071 ] {} AsyncLoader: Change current priority: none -> 2 Stdout:2025.04.02 03:32:19.378752 [ 6071 ] {} AsyncLoader: Spawn loader worker #1 in BackgrndStartup Stdout:2025.04.02 03:32:19.378800 [ 6071 ] {} UserDefinedSQLObjectsLoaderFromDisk: Loading user defined objects from /var/lib/clickhouse/user_defined/ Stdout:2025.04.02 03:32:19.378811 [ 6071 ] {} UserDefinedSQLObjectsLoaderFromDisk: The directory for user defined objects (/var/lib/clickhouse/user_defined/) does not exist: nothing to load Stdout:2025.04.02 03:32:19.378842 [ 6071 ] {} WorkloadEntityDiskStorage: Loading workload entities from /var/lib/clickhouse/workload/ Stdout:2025.04.02 03:32:19.378848 [ 6071 ] {} WorkloadEntityDiskStorage: The directory for workload entities (/var/lib/clickhouse/workload/) does not exist: nothing to load Stdout:2025.04.02 03:32:19.378856 [ 6755 ] {} AsyncLoader: Execute load job 'startup Ordinary database default' in BackgrndStartup Stdout:2025.04.02 03:32:19.378891 [ 6755 ] {} AsyncLoader: Finish load job 'startup Ordinary database default' with status OK Stdout:2025.04.02 03:32:19.378897 [ 6755 ] {} AsyncLoader: Spawn loader worker #2 in BackgrndStartup Stdout:2025.04.02 03:32:19.378909 [ 6755 ] {} AsyncLoader: Execute load job 'startup Atomic database default' in BackgrndStartup Stdout:2025.04.02 03:32:19.378914 [ 6755 ] {} AsyncLoader: Finish load job 'startup Atomic database default' with status OK Stdout:2025.04.02 03:32:19.378922 [ 6755 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:19.378947 [ 6754 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:19.379246 [ 6754 ] {} AsyncLoader: Change current priority: 2 -> none Stdout:2025.04.02 03:32:19.379647 [ 6071 ] {} ZooKeeperClient: Adding ZooKeeper host zoo2:2181 (172.16.8.4:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:19.380162 [ 6071 ] {} ZooKeeperClient: Adding ZooKeeper host zoo1:2181 (172.16.8.2:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:19.380782 [ 6071 ] {} ZooKeeperClient: Adding ZooKeeper host zoo3:2181 (172.16.8.3:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:19.382996 [ 6071 ] {} ZooKeeperClient: Connected to ZooKeeper at 172.16.8.4:2181 with session_id 13 Stdout:2025.04.02 03:32:19.385572 [ 6071 ] {} ZooKeeperClient: Keeper feature flag FILTERED_LIST: enabled Stdout:2025.04.02 03:32:19.385586 [ 6071 ] {} ZooKeeperClient: Keeper feature flag MULTI_READ: disabled Stdout:2025.04.02 03:32:19.385591 [ 6071 ] {} ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: enabled Stdout:2025.04.02 03:32:19.385601 [ 6071 ] {} ZooKeeperClient: Keeper feature flag CREATE_IF_NOT_EXISTS: disabled Stdout:2025.04.02 03:32:19.385606 [ 6071 ] {} ZooKeeperClient: Keeper feature flag REMOVE_RECURSIVE: enabled Stdout:2025.04.02 03:32:19.385614 [ 6071 ] {} ZooKeeper: Initialized, hosts: zoo1:2181,zoo2:2181,zoo3:2181 Stdout:2025.04.02 03:32:19.388130 [ 6071 ] {} Application: Loaded metadata. Stdout:2025.04.02 03:32:19.388259 [ 6071 ] {} bool DB::(anonymous namespace)::checkPermissionsImpl(): Code: 412. DB::Exception: Can't receive Netlink response: error -2. (NETLINK_ERROR) (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:19.388284 [ 6071 ] {} Application: Tasks stats provider: procfs Stdout:2025.04.02 03:32:19.388383 [ 6071 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:19.388393 [ 6071 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:19.388761 [ 6071 ] {} MySQLHandlerFactory: Failed to create SSL context. SSL will be disabled. Error: Poco::Exception. Code: 1000, e.code() = 0, SSL Exception: Configuration error: no certificate file has been specified (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:19.388815 [ 6071 ] {} MySQLHandlerFactory: Failed to read RSA key pair from server certificate. Error: Code: 139. DB::Exception: Certificate file is not set. (NO_ELEMENTS_IN_CONFIG) (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:19.388827 [ 6071 ] {} MySQLHandlerFactory: Generating new RSA key pair. Stdout:2025.04.02 03:32:19.478069 [ 6071 ] {} CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. Stdout:2025.04.02 03:32:19.478089 [ 6071 ] {} CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. Docker volumes for project roottestreloadquerymaskingrules-gw7 are DRIVER VOLUME NAME Stdout:2025.04.02 03:32:19.480781 [ 6071 ] {} AsyncLoader: Schedule load job 'startup ddl worker' into BackgrndStartup Stdout:2025.04.02 03:32:19.480794 [ 6071 ] {} AsyncLoader: Change current priority: none -> 2 Cleanup called Stdout:2025.04.02 03:32:19.480798 [ 6071 ] {} AsyncLoader: Spawn loader worker #1 in BackgrndStartup Stdout:2025.04.02 03:32:19.480879 [ 6071 ] {} Application: Listening for http://0.0.0.0:8123 Stdout:2025.04.02 03:32:19.480879 [ 6753 ] {} AsyncLoader: Execute load job 'startup ddl worker' in BackgrndStartup Stdout:2025.04.02 03:32:19.481049 [ 6746 ] {} DDLWorker: Starting DDLWorker thread Stdout:2025.04.02 03:32:19.481061 [ 6746 ] {} DDLWorker: Initializing DDLWorker thread Stdout:2025.04.02 03:32:19.481072 [ 6071 ] {} Application: Listening for native protocol (tcp): 0.0.0.0:9000 Stdout:2025.04.02 03:32:19.481173 [ 6071 ] {} Application: Listening for MySQL compatibility protocol: 0.0.0.0:9004 Stdout:2025.04.02 03:32:19.481262 [ 6071 ] {} Application: Listening for PostgreSQL compatibility protocol: 0.0.0.0:9005 Stdout:2025.04.02 03:32:19.481276 [ 6071 ] {} Application: Ready for connections. Stdout:2025.04.02 03:32:19.481466 [ 6753 ] {} AsyncLoader: Finish load job 'startup ddl worker' with status OK Stdout:2025.04.02 03:32:19.481665 [ 6753 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:19.481683 [ 6753 ] {} AsyncLoader: Change current priority: 2 -> none Stdout:2025.04.02 03:32:19.481703 [ 6753 ] {} DDLWorker: Started DDLWorker cleanup thread Stdout:2025.04.02 03:32:19.485486 [ 6746 ] {} DDLWorker: Initialized DDLWorker thread Stdout:2025.04.02 03:32:19.485579 [ 6753 ] {} DDLWorker: Cleaning queue Stdout:2025.04.02 03:32:19.485925 [ 6746 ] {} DDLWorker: Scheduling tasks Stdout:2025.04.02 03:32:19.485947 [ 6746 ] {} DDLWorker: Don't have unfinished tasks after restarting Stdout:2025.04.02 03:32:19.486265 [ 6746 ] {} DDLWorker: scheduleTasks: initialized=true, size_before_filtering=0, queue_size=0, entries=none..none, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=none Stdout:2025.04.02 03:32:19.486279 [ 6746 ] {} DDLWorker: No tasks to schedule Stdout:2025.04.02 03:32:19.486285 [ 6746 ] {} DDLWorker: Waiting for queue updates Stdout:2025.04.02 03:32:20.102142 [ 6084 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57462 Stdout:2025.04.02 03:32:20.102228 [ 6084 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:20.102247 [ 6084 ] {} TCP-Session-c34abbff-a0df-47af-a8f5-d06b00e8d7fc: Authenticating user 'default' from 172.16.8.1:57462 Stdout:2025.04.02 03:32:20.102283 [ 6084 ] {} TCP-Session-c34abbff-a0df-47af-a8f5-d06b00e8d7fc: c34abbff-a0df-47af-a8f5-d06b00e8d7fc Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:20.102289 [ 6084 ] {} TCP-Session-c34abbff-a0df-47af-a8f5-d06b00e8d7fc: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:20.102379 [ 6084 ] {} ContextAccess (default): Settings: readonly = 0, allow_ddl = true, allow_introspection_functions = false Stdout:2025.04.02 03:32:20.102406 [ 6084 ] {} ContextAccess (default): List of all grants: GRANT CHECK, SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, ROLE ADMIN, CREATE ROW POLICY, ALTER ROW POLICY, DROP ROW POLICY, CREATE QUOTA, ALTER QUOTA, DROP QUOTA, CREATE SETTINGS PROFILE, ALTER SETTINGS PROFILE, DROP SETTINGS PROFILE, ALLOW SQL SECURITY NONE, SHOW ACCESS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* WITH GRANT OPTION, GRANT TABLE ENGINE ON * WITH GRANT OPTION, GRANT CREATE USER, ALTER USER, DROP USER, CREATE ROLE, ALTER ROLE, DROP ROLE, SET DEFINER ON * WITH GRANT OPTION, GRANT ALTER NAMED COLLECTION, CREATE NAMED COLLECTION, DROP NAMED COLLECTION, SHOW NAMED COLLECTIONS, NAMED COLLECTION ON * WITH GRANT OPTION Stdout:2025.04.02 03:32:20.102417 [ 6084 ] {} ContextAccess (default): List of all grants including implicit: GRANT CHECK, SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, ROLE ADMIN, CREATE ROW POLICY, ALTER ROW POLICY, DROP ROW POLICY, CREATE QUOTA, ALTER QUOTA, DROP QUOTA, CREATE SETTINGS PROFILE, ALTER SETTINGS PROFILE, DROP SETTINGS PROFILE, ALLOW SQL SECURITY NONE, SHOW ACCESS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* WITH GRANT OPTION, GRANT TABLE ENGINE ON * WITH GRANT OPTION, GRANT CREATE USER, ALTER USER, DROP USER, CREATE ROLE, ALTER ROLE, DROP ROLE, SET DEFINER ON * WITH GRANT OPTION, GRANT ALTER NAMED COLLECTION, CREATE NAMED COLLECTION, DROP NAMED COLLECTION, SHOW NAMED COLLECTIONS, NAMED COLLECTION ON * WITH GRANT OPTION Stdout:2025.04.02 03:32:20.111283 [ 6084 ] {} TCP-Session-c34abbff-a0df-47af-a8f5-d06b00e8d7fc: c34abbff-a0df-47af-a8f5-d06b00e8d7fc Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:20.111576 [ 6084 ] {86001674-9dcd-44e0-9868-86db942ce2f3} executeQuery: (from 172.16.8.1:57462) (query 1, line 1) select 20 (stage: Complete) Stdout:2025.04.02 03:32:20.111745 [ 6084 ] {86001674-9dcd-44e0-9868-86db942ce2f3} Planner: Query to stage Complete Stdout:2025.04.02 03:32:20.111814 [ 6084 ] {86001674-9dcd-44e0-9868-86db942ce2f3} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:20.112484 [ 6084 ] {86001674-9dcd-44e0-9868-86db942ce2f3} executeQuery: Read 1 rows, 1.00 B in 0.000909 sec., 1100.1100110011002 rows/sec., 1.07 KiB/sec. Stdout:2025.04.02 03:32:20.112572 [ 6084 ] {86001674-9dcd-44e0-9868-86db942ce2f3} TCPHandler: Processed in 0.001455209 sec. Stdout:2025.04.02 03:32:20.124329 [ 6084 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:20.124362 [ 6084 ] {} TCP-Session-c34abbff-a0df-47af-a8f5-d06b00e8d7fc: c34abbff-a0df-47af-a8f5-d06b00e8d7fc Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.282478 [ 6084 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57464 Stdout:2025.04.02 03:32:22.282521 [ 6084 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:22.282534 [ 6084 ] {} TCP-Session-70feb657-9341-4860-a3a5-7e85d57fa56d: Authenticating user 'default' from 172.16.8.1:57464 Stdout:2025.04.02 03:32:22.282558 [ 6084 ] {} TCP-Session-70feb657-9341-4860-a3a5-7e85d57fa56d: 70feb657-9341-4860-a3a5-7e85d57fa56d Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.282564 [ 6084 ] {} TCP-Session-70feb657-9341-4860-a3a5-7e85d57fa56d: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.293392 [ 6084 ] {} TCP-Session-70feb657-9341-4860-a3a5-7e85d57fa56d: 70feb657-9341-4860-a3a5-7e85d57fa56d Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:22.293648 [ 6084 ] {54d1d810-a453-4e7e-ba3e-f0c547b57d7e} executeQuery: (from 172.16.8.1:57464) (query 1, line 2) CREATE TABLE test_remove_disk_from_policy ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' (stage: Complete) Stdout:2025.04.02 03:32:22.293928 [ 6084 ] {54d1d810-a453-4e7e-ba3e-f0c547b57d7e} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:22.294546 [ 6084 ] {54d1d810-a453-4e7e-ba3e-f0c547b57d7e} default.test_remove_disk_from_policy (ee9c3908-42fe-4bec-94ec-e0eae1e532db): Loading data parts Stdout:2025.04.02 03:32:22.295081 [ 6084 ] {54d1d810-a453-4e7e-ba3e-f0c547b57d7e} default.test_remove_disk_from_policy (ee9c3908-42fe-4bec-94ec-e0eae1e532db): There are no data parts Stdout:2025.04.02 03:32:22.295188 [ 6084 ] {54d1d810-a453-4e7e-ba3e-f0c547b57d7e} AsyncLoader: Prioritize load job 'startup Atomic database default': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:22.295197 [ 6084 ] {54d1d810-a453-4e7e-ba3e-f0c547b57d7e} AsyncLoader: Prioritize load job 'startup Ordinary database default': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:22.298844 [ 6084 ] {54d1d810-a453-4e7e-ba3e-f0c547b57d7e} DatabaseAtomic (default): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:22.298859 [ 6084 ] {54d1d810-a453-4e7e-ba3e-f0c547b57d7e} DatabaseAtomic (default): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:22.299203 [ 6094 ] {} default.test_remove_disk_from_policy (ee9c3908-42fe-4bec-94ec-e0eae1e532db): Didn't start merge: There are no parts that can be merged. (Collector returned empty ranges set) Stdout:2025.04.02 03:32:22.299254 [ 6084 ] {54d1d810-a453-4e7e-ba3e-f0c547b57d7e} TCPHandler: Processed in 0.006042461 sec. Stdout:2025.04.02 03:32:22.299643 [ 6084 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:22.299661 [ 6084 ] {} TCP-Session-70feb657-9341-4860-a3a5-7e85d57fa56d: 70feb657-9341-4860-a3a5-7e85d57fa56d Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.345275 [ 6084 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57472 Stdout:2025.04.02 03:32:22.345317 [ 6084 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:22.345338 [ 6084 ] {} TCP-Session-3e8055d8-7c72-4a31-8f0b-5ae69284fd1a: Authenticating user 'default' from 172.16.8.1:57472 Stdout:2025.04.02 03:32:22.345357 [ 6084 ] {} TCP-Session-3e8055d8-7c72-4a31-8f0b-5ae69284fd1a: 3e8055d8-7c72-4a31-8f0b-5ae69284fd1a Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.345362 [ 6084 ] {} TCP-Session-3e8055d8-7c72-4a31-8f0b-5ae69284fd1a: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.358378 [ 6084 ] {} TCP-Session-3e8055d8-7c72-4a31-8f0b-5ae69284fd1a: 3e8055d8-7c72-4a31-8f0b-5ae69284fd1a Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:22.358604 [ 6084 ] {8a60d840-3182-420b-b7ba-4900bb3f7315} executeQuery: (from 172.16.8.1:57472) (query 1, line 1) SELECT volume_name FROM system.storage_policies WHERE policy_name = 'test_remove_disk_from_policy_cool_policy' (stage: Complete) Stdout:2025.04.02 03:32:22.358842 [ 6084 ] {8a60d840-3182-420b-b7ba-4900bb3f7315} Planner: Query to stage Complete Stdout:2025.04.02 03:32:22.358972 [ 6084 ] {8a60d840-3182-420b-b7ba-4900bb3f7315} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:22.359914 [ 6084 ] {8a60d840-3182-420b-b7ba-4900bb3f7315} executeQuery: Read 16 rows, 1.60 KiB in 0.001355 sec., 11808.118081180812 rows/sec., 1.15 MiB/sec. Stdout:2025.04.02 03:32:22.360012 [ 6084 ] {8a60d840-3182-420b-b7ba-4900bb3f7315} TCPHandler: Processed in 0.001758848 sec. Stdout:2025.04.02 03:32:22.371841 [ 6084 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:22.371861 [ 6084 ] {} TCP-Session-3e8055d8-7c72-4a31-8f0b-5ae69284fd1a: 3e8055d8-7c72-4a31-8f0b-5ae69284fd1a Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.403222 [ 6084 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57476 Stdout:2025.04.02 03:32:22.403266 [ 6084 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:22.403278 [ 6084 ] {} TCP-Session-4d053257-6d11-48ba-8a52-548c3891e9db: Authenticating user 'default' from 172.16.8.1:57476 Stdout:2025.04.02 03:32:22.403305 [ 6084 ] {} TCP-Session-4d053257-6d11-48ba-8a52-548c3891e9db: 4d053257-6d11-48ba-8a52-548c3891e9db Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.403315 [ 6084 ] {} TCP-Session-4d053257-6d11-48ba-8a52-548c3891e9db: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.411915 [ 6084 ] {} TCP-Session-4d053257-6d11-48ba-8a52-548c3891e9db: 4d053257-6d11-48ba-8a52-548c3891e9db Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:22.412133 [ 6084 ] {56afac94-2f52-4fb8-bd49-2a690dc03a62} executeQuery: (from 172.16.8.1:57476) (query 1, line 1) SELECT disks FROM system.storage_policies WHERE policy_name = 'test_remove_disk_from_policy_cool_policy' (stage: Complete) Stdout:2025.04.02 03:32:22.412346 [ 6084 ] {56afac94-2f52-4fb8-bd49-2a690dc03a62} Planner: Query to stage Complete Stdout:2025.04.02 03:32:22.412461 [ 6084 ] {56afac94-2f52-4fb8-bd49-2a690dc03a62} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:22.413671 [ 6084 ] {56afac94-2f52-4fb8-bd49-2a690dc03a62} executeQuery: Read 16 rows, 1.60 KiB in 0.00151 sec., 10596.026490066224 rows/sec., 1.03 MiB/sec. Stdout:2025.04.02 03:32:22.413980 [ 6084 ] {56afac94-2f52-4fb8-bd49-2a690dc03a62} TCPHandler: Processed in 0.002153137 sec. Stdout:2025.04.02 03:32:22.423365 [ 6084 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:22.423389 [ 6084 ] {} TCP-Session-4d053257-6d11-48ba-8a52-548c3891e9db: 4d053257-6d11-48ba-8a52-548c3891e9db Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.470321 [ 6084 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57478 Stdout:2025.04.02 03:32:22.470364 [ 6084 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:22.470379 [ 6084 ] {} TCP-Session-561cbde4-1d7d-4c53-aeb9-625745c74ad2: Authenticating user 'default' from 172.16.8.1:57478 Stdout:2025.04.02 03:32:22.470402 [ 6084 ] {} TCP-Session-561cbde4-1d7d-4c53-aeb9-625745c74ad2: 561cbde4-1d7d-4c53-aeb9-625745c74ad2 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.470412 [ 6084 ] {} TCP-Session-561cbde4-1d7d-4c53-aeb9-625745c74ad2: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.479365 [ 6084 ] {} TCP-Session-561cbde4-1d7d-4c53-aeb9-625745c74ad2: 561cbde4-1d7d-4c53-aeb9-625745c74ad2 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:22.479562 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} executeQuery: (from 172.16.8.1:57478) (query 1, line 1) SYSTEM RELOAD CONFIG (stage: Complete) Stdout:2025.04.02 03:32:22.481385 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigReloader: Loading config '/etc/clickhouse-server/config.xml' Stdout:2025.04.02 03:32:22.481398 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/config.xml'. Stdout:2025.04.02 03:32:22.482223 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:22.482306 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:22.482379 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_enable_keeper_async_replication.xml'. Stdout:2025.04.02 03:32:22.482435 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_instance_config.xml'. Stdout:2025.04.02 03:32:22.482584 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/logs_config.xml'. Stdout:2025.04.02 03:32:22.482670 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/storage_configuration.xml'. Stdout:2025.04.02 03:32:22.486936 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'. Stdout:2025.04.02 03:32:22.487028 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performing update on configuration Stdout:2025.04.02 03:32:22.493403 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} Application: Changed setting 'max_server_memory_usage' to 27.54 GiB (30.60 GiB available memory * 0.90 max_server_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:22.493421 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.30 GiB (30.60 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:22.493426 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} Application: Merges and mutations memory limit is set to 15.30 GiB Stdout:2025.04.02 03:32:22.494426 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} Application: Setting max_remote_read_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:22.494435 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} Application: Setting max_remote_write_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:22.494441 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} Application: ConcurrencyControl limit is set to 32 CPU slots with 'fair_round_robin' scheduler Stdout:2025.04.02 03:32:22.494458 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} MergeTreeBackgroundExecutor: Loaded new threads count (16) and max tasks count (32) for MergeMutateExecutor Stdout:2025.04.02 03:32:22.494471 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for MoveExecutor Stdout:2025.04.02 03:32:22.494475 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} MergeTreeBackgroundExecutor: Loaded new threads count (16) and max tasks count (16) for FetchExecutor Stdout:2025.04.02 03:32:22.494479 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for CommonExecutor Stdout:2025.04.02 03:32:22.494996 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicy (small_jbod_with_external): Storage policy small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:22.495004 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicySelector: Storage policy `small_jbod_with_external` loaded Stdout:2025.04.02 03:32:22.495125 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicy (one_more_small_jbod_with_external): Storage policy one_more_small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:22.495130 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicySelector: Storage policy `one_more_small_jbod_with_external` loaded Stdout:2025.04.02 03:32:22.495253 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicy (jbods_with_external): Storage policy jbods_with_external created, total volumes 2 Stdout:2025.04.02 03:32:22.495257 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicySelector: Storage policy `jbods_with_external` loaded Stdout:2025.04.02 03:32:22.495369 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicy (moving_jbod_with_external): Storage policy moving_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:22.495374 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicySelector: Storage policy `moving_jbod_with_external` loaded Stdout:2025.04.02 03:32:22.495431 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StorageConfiguration: Volume `small` max_data_part_size is too low (2.00 MiB < 8.00 MiB) Stdout:2025.04.02 03:32:22.495509 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicy (default_disk_with_external): Storage policy default_disk_with_external created, total volumes 2 Stdout:2025.04.02 03:32:22.495513 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicySelector: Storage policy `default_disk_with_external` loaded Stdout:2025.04.02 03:32:22.495670 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StorageConfiguration: Volume `special_warning_small_volume` max_data_part_size is too low (1.00 KiB < 8.00 MiB) Stdout:2025.04.02 03:32:22.495747 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicy (special_warning_policy): Storage policy special_warning_policy created, total volumes 4 Stdout:2025.04.02 03:32:22.495752 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicySelector: Storage policy `special_warning_policy` loaded Stdout:2025.04.02 03:32:22.495815 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicy (cool_policy): Storage policy cool_policy created, total volumes 1 Stdout:2025.04.02 03:32:22.495819 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicySelector: Storage policy `cool_policy` loaded Stdout:2025.04.02 03:32:22.495833 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} StoragePolicy (default): Storage policy default created, total volumes 1 Stdout:2025.04.02 03:32:22.495909 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} Context: An error has occurred while reloading storage policies, storage policies were not applied: Storage policy `test_remove_disk_from_policy_cool_policy` is missing in new configuration Stdout:2025.04.02 03:32:22.496047 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration Stdout:2025.04.02 03:32:22.497792 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigReloader: Loading config '/etc/clickhouse-server/users.xml' Stdout:2025.04.02 03:32:22.498550 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'. Stdout:2025.04.02 03:32:22.498569 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performing update on configuration Stdout:2025.04.02 03:32:22.499053 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performed update on configuration Stdout:2025.04.02 03:32:22.499431 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} MemoryTracker: Query peak memory usage: 10.28 MiB. Stdout:2025.04.02 03:32:22.499478 [ 6084 ] {83efb0fb-a570-4afb-b331-d6a82b51ace1} TCPHandler: Processed in 0.020232789 sec. Stdout:2025.04.02 03:32:22.499799 [ 6084 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:22.499821 [ 6084 ] {} TCP-Session-561cbde4-1d7d-4c53-aeb9-625745c74ad2: 561cbde4-1d7d-4c53-aeb9-625745c74ad2 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.536425 [ 6084 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57486 Stdout:2025.04.02 03:32:22.536466 [ 6084 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:22.536481 [ 6084 ] {} TCP-Session-89b4820e-815f-448c-a32f-0ddf8e6acc9f: Authenticating user 'default' from 172.16.8.1:57486 Stdout:2025.04.02 03:32:22.536504 [ 6084 ] {} TCP-Session-89b4820e-815f-448c-a32f-0ddf8e6acc9f: 89b4820e-815f-448c-a32f-0ddf8e6acc9f Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.536511 [ 6084 ] {} TCP-Session-89b4820e-815f-448c-a32f-0ddf8e6acc9f: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.545869 [ 6084 ] {} TCP-Session-89b4820e-815f-448c-a32f-0ddf8e6acc9f: 89b4820e-815f-448c-a32f-0ddf8e6acc9f Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:22.546099 [ 6084 ] {10d5268c-75f1-42ad-a550-a2e4e65586a3} executeQuery: (from 172.16.8.1:57486) (query 1, line 1) SELECT volume_name FROM system.storage_policies WHERE policy_name = 'test_remove_disk_from_policy_cool_policy' (stage: Complete) Stdout:2025.04.02 03:32:22.546333 [ 6084 ] {10d5268c-75f1-42ad-a550-a2e4e65586a3} Planner: Query to stage Complete Stdout:2025.04.02 03:32:22.546455 [ 6084 ] {10d5268c-75f1-42ad-a550-a2e4e65586a3} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:22.547385 [ 6084 ] {10d5268c-75f1-42ad-a550-a2e4e65586a3} executeQuery: Read 16 rows, 1.60 KiB in 0.001316 sec., 12158.0547112462 rows/sec., 1.19 MiB/sec. Stdout:2025.04.02 03:32:22.547585 [ 6084 ] {10d5268c-75f1-42ad-a550-a2e4e65586a3} TCPHandler: Processed in 0.00183513 sec. Stdout:2025.04.02 03:32:22.558656 [ 6084 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:22.558681 [ 6084 ] {} TCP-Session-89b4820e-815f-448c-a32f-0ddf8e6acc9f: 89b4820e-815f-448c-a32f-0ddf8e6acc9f Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.642383 [ 6084 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:57490 Stdout:2025.04.02 03:32:22.642430 [ 6084 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:22.642446 [ 6084 ] {} TCP-Session-28a43d51-751b-4034-b100-1545a9f77cab: Authenticating user 'default' from 172.16.8.1:57490 Stdout:2025.04.02 03:32:22.642469 [ 6084 ] {} TCP-Session-28a43d51-751b-4034-b100-1545a9f77cab: 28a43d51-751b-4034-b100-1545a9f77cab Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.642483 [ 6084 ] {} TCP-Session-28a43d51-751b-4034-b100-1545a9f77cab: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:22.654392 [ 6084 ] {} TCP-Session-28a43d51-751b-4034-b100-1545a9f77cab: 28a43d51-751b-4034-b100-1545a9f77cab Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:22.654624 [ 6084 ] {6683cf4c-4fdb-4553-a2bd-a14ec31b4d45} executeQuery: (from 172.16.8.1:57490) (query 1, line 1) SELECT disks FROM system.storage_policies WHERE policy_name = 'test_remove_disk_from_policy_cool_policy' (stage: Complete) Stdout:2025.04.02 03:32:22.654839 [ 6084 ] {6683cf4c-4fdb-4553-a2bd-a14ec31b4d45} Planner: Query to stage Complete Stdout:2025.04.02 03:32:22.654958 [ 6084 ] {6683cf4c-4fdb-4553-a2bd-a14ec31b4d45} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:22.655764 [ 6084 ] {6683cf4c-4fdb-4553-a2bd-a14ec31b4d45} executeQuery: Read 16 rows, 1.60 KiB in 0.0012 sec., 13333.333333333334 rows/sec., 1.30 MiB/sec. Stdout:2025.04.02 03:32:22.655865 [ 6084 ] {6683cf4c-4fdb-4553-a2bd-a14ec31b4d45} TCPHandler: Processed in 0.001592636 sec. Stdout:2025.04.02 03:32:22.667172 [ 6084 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:22.667194 [ 6084 ] {} TCP-Session-28a43d51-751b-4034-b100-1545a9f77cab: 28a43d51-751b-4034-b100-1545a9f77cab Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db thread 1: insert for 2000-01-02: [6, 5, 8, 10, 7] Executing query DROP TABLE IF EXISTS test_remove_disk_from_policy on node1 Executing query INSERT INTO test_mutations FORMAT TSV on node5 Docker networks for project roottestreloadquerymaskingrules-gw7 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreloadquerymaskingrules-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Docker volumes for project roottestreloadquerymaskingrules-gw7 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreloadquerymaskingrules-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreloadquerymaskingrules-gw7 Trying to prune unused networks... Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 [gw2] PASSED test_reloading_storage_configuration/test.py::test_remove_disk_from_policy run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] Trying to prune unused images... Command:[docker image prune -f] test_reloading_storage_configuration/test.py::test_remove_policy Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout: PID TTY TIME CMD Stdout: 6071 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] Stdout:9 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 9 Setup directory for instance: node Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files [] to /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/node/configs/config.d Setup database dir /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/node/database Setup logs dir /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/node/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/.env --project-name roottestreloadquerymaskingrules-gw7 --file /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/node/docker-compose.yml pull] Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:6071 thread 0: insert for 2000-01-01: [2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [6, 9, 4, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query drop table test_drop_table sync on node2 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo2 to foo3 on node2 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo2 to foo3 on node2 Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo2 to foo3 on node2 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo3 to num2 on node1 thread 0: insert for 2000-01-01: [4, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select count(n), sum(n) from test_drop_table on node1 thread 1: insert for 2000-01-02: [8, 5, 5, 5, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance Executing query drop table test_drop_table sync on node1 Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopped Starting zookeeper node: zoo1 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo1] Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance thread 0: insert for 2000-01-01: [3, 10, 7, 8, 2, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [1, 9, 3, 5, 6, 2, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Started Starting zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo2] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 thread 0: insert for 2000-01-01: [4, 2, 8, 8, 3, 6, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP TABLE IF EXISTS limited_sends2 SYNC on node1 http://localhost:None "POST /v1.46/exec/80ff5e9e6d6e81d2beee377ddee5106b698b95e9c9cd71a476295d9e6b12b505/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/80ff5e9e6d6e81d2beee377ddee5106b698b95e9c9cd71a476295d9e6b12b505/json HTTP/1.1" 200 586 thread 1: insert for 2000-01-02: [2, 3, 8, 9, 2, 4, 9, 6, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Started Starting zookeeper node: zoo3 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo3] Executing query DROP TABLE IF EXISTS limited_sends3 SYNC on node1 thread 0: insert for 2000-01-01: [10, 5, 3, 10, 7, 1, 7, 2, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [8, 8, 3, 4, 5, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Started run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo -n > /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c echo -n > /var/log/clickhouse-server/clickhouse-server.log] run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo -n > /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c echo -n > /var/log/clickhouse-server/clickhouse-server.log] get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.2.2, port:2281, use_ssl:True Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused thread 0: insert for 2000-01-01: [1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Executing query DROP TABLE IF EXISTS s3_test SYNC on node1 [gw8] PASSED test_replicated_merge_tree_s3_zero_copy/test.py::test_drop_table thread 1: insert for 2000-01-02: [6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node1 Executing query DROP TABLE IF EXISTS s3_test SYNC on node2 Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node2 Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Executing query DROP TABLE IF EXISTS s3_test SYNC on node3 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance thread 0: insert for 2000-01-01: [1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node3 thread 1: insert for 2000-01-02: [2, 3, 1, 10, 6, 10, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:6846 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Resetting dropped connection: 172.16.9.7 http://172.16.9.7:9001 "GET /root?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=data%2F HTTP/1.1" 200 0 test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[0-16] Executing query CREATE TABLE s3_test ON CLUSTER cluster ( dt Date, id Int64, data String, INDEX min_max (id) TYPE minmax GRANULARITY 3 ) ENGINE=ReplicatedMergeTree() PARTITION BY dt ORDER BY (dt, id) SETTINGS storage_policy='s3' ,min_rows_for_wide_part=0 on node1 Stdout:6846 Executing query select 20 on node1 Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Stopped Stopping zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadzookeeper-gw5 --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop zoo2] Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [10, 8, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO s3_test VALUES ('2020-01-01',1,'WkobbEkNMp'),('2020-01-01',2,'FVCEAQuRoL'),('2020-01-01',3,'tLndcWDAia'),('2020-01-01',4,'XLeQaCnToE'),('2020-01-01',5,'kVhLxJvKNx'),('2020-01-01',6,'kfaYcgZiIt'),('2020-01-01',7,'JBHlSNSLAs'),('2020-01-01',8,'bYRBPTiarS'),('2020-01-01',9,'AaSsdXwpdH'),('2020-01-01',10,'YlGNpOYKoy'),('2020-01-01',11,'ZhmEjHsSdY'),('2020-01-01',12,'jFdwcnBrYW'),('2020-01-01',13,'eRnLOqTkKu'),('2020-01-01',14,'GJUARccdiI'),('2020-01-01',15,'qTIXfcYjrU'),('2020-01-01',16,'HInfdDjCYr'),('2020-01-01',17,'cTKrCdxZSp'),('2020-01-01',18,'oIsHaemlfN'),('2020-01-01',19,'LnXyHSRHek'),('2020-01-01',20,'dTMbUoPBBD'),('2020-01-01',21,'CiNSNVNDfH'),('2020-01-01',22,'sLCQVjtdio'),('2020-01-01',23,'UHRfFzQanW'),('2020-01-01',24,'IjhuqIPHiB'),('2020-01-01',25,'feJqFqLiJA'),('2020-01-01',26,'mthOiDpqlZ'),('2020-01-01',27,'facRBcobyI'),('2020-01-01',28,'XNBabHppzs'),('2020-01-01',29,'oHhSiGcYEx'),('2020-01-01',30,'TMvboKSpDP'),('2020-01-01',31,'fVExznzLDR'),('2020-01-01',32,'NUx on node1 thread 0: insert for 2000-01-01: [6, 5, 9, 9, 4, 9, 1, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO s3_test VALUES ('2020-01-02',1,'KEgzmPVWTj'),('2020-01-02',2,'GLfYzpYIPk'),('2020-01-02',3,'aiaEhMAByw'),('2020-01-02',4,'lPstTRiktB'),('2020-01-02',5,'DuNqogGXPd'),('2020-01-02',6,'ITSuhlqtCv'),('2020-01-02',7,'mJiUdbmbrs'),('2020-01-02',8,'DGZdJxWMtv'),('2020-01-02',9,'nxaNmfVfrr'),('2020-01-02',10,'dOGSVYsbVy'),('2020-01-02',11,'FnawRkvFyh'),('2020-01-02',12,'PvSrrdWWew'),('2020-01-02',13,'kmZMXcsYTz'),('2020-01-02',14,'LYjoooOvpk'),('2020-01-02',15,'vhotWtoaNK'),('2020-01-02',16,'PKqhIbZzhI'),('2020-01-02',17,'XYnRtKsmLP'),('2020-01-02',18,'IOakmlNpSr'),('2020-01-02',19,'TWMcpqdNbl'),('2020-01-02',20,'wUHNGnfQBH'),('2020-01-02',21,'nlHGzJPzCR'),('2020-01-02',22,'xoPwUrahWo'),('2020-01-02',23,'tMUfjZkYys'),('2020-01-02',24,'vdFVHznxPc'),('2020-01-02',25,'WeoWDDVGjE'),('2020-01-02',26,'jUocOfLViK'),('2020-01-02',27,'VxtYSxjxaw'),('2020-01-02',28,'aaJlFgxRqs'),('2020-01-02',29,'EwMJANrWjT'),('2020-01-02',30,'HcIjHnycON'),('2020-01-02',31,'rBccGHUMBq'),('2020-01-02',32,'hsU on node2 Executing query INSERT INTO s3_test VALUES ('2020-01-03',1,'ftSlwEqrGE'),('2020-01-03',2,'xLkrVMwOBF'),('2020-01-03',3,'DHJwZKyGLH'),('2020-01-03',4,'xMtUdQMtYI'),('2020-01-03',5,'KBFbgmUCzn'),('2020-01-03',6,'MEOFhbjgZJ'),('2020-01-03',7,'nEbJHecAXt'),('2020-01-03',8,'yDKGfDRuwg'),('2020-01-03',9,'VKEuStXwUd'),('2020-01-03',10,'KlXudARFPY'),('2020-01-03',11,'bbntQWUZrP'),('2020-01-03',12,'nOtoBfCnHr'),('2020-01-03',13,'IgQIQDUbiR'),('2020-01-03',14,'DTvnqPwucU'),('2020-01-03',15,'tpNhNgFPCy'),('2020-01-03',16,'wKEgpjIizM'),('2020-01-03',17,'YcVDryRMny'),('2020-01-03',18,'FqswPSIJrE'),('2020-01-03',19,'CyRsjefUtI'),('2020-01-03',20,'UmZYmSSxaZ'),('2020-01-03',21,'xbsxnOzkbG'),('2020-01-03',22,'qfTYFZxZFG'),('2020-01-03',23,'TXjceADPeY'),('2020-01-03',24,'XmYtTvQjYF'),('2020-01-03',25,'TdRnNiFJxb'),('2020-01-03',26,'wMOjPpGenr'),('2020-01-03',27,'vvoHCpomYX'),('2020-01-03',28,'ClrhcRmrza'),('2020-01-03',29,'cCONuBvTxH'),('2020-01-03',30,'WCrXrDhnCQ'),('2020-01-03',31,'PNqRgyYDjd'),('2020-01-03',32,'PTI on node3 thread 0: insert for 2000-01-01: [6, 8, 8, 9, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT * FROM s3_test order by dt, id FORMAT Values on node1 thread 1: insert for 2000-01-02: [10, 3, 2, 5, 8, 8, 10, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT * FROM s3_test order by dt, id FORMAT Values on node2 Executing query SELECT * FROM s3_test order by dt, id FORMAT Values on node3 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Connection dropped: socket connection broken Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance thread 0: insert for 2000-01-01: [10, 7, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 http://172.16.9.7:9001 "GET /root?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=data%2F HTTP/1.1" 200 0 Executing query DROP TABLE IF EXISTS s3_test SYNC on node1 [gw8] PASSED test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[0-16] Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Stopped Stopping zookeeper node: zoo3 Command:[docker compose --project-name roottestreloadzookeeper-gw5 --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop zoo3] Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node1 Executing query DROP TABLE IF EXISTS limited_sends4 SYNC on node1 Executing query DROP TABLE IF EXISTS s3_test SYNC on node2 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [10, 8, 6, 5, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node2 Executing query DROP TABLE IF EXISTS s3_test SYNC on node3 thread 0: insert for 2000-01-01: [1, 8, 10, 5, 9, 3, 7, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node3 thread 1: insert for 2000-01-02: [6, 6, 3, 5, 5, 2, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 http://172.16.9.7:9001 "GET /root?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=data%2F HTTP/1.1" 200 0 Executing query CREATE TABLE s3_test ON CLUSTER cluster ( dt Date, id Int64, data String, INDEX min_max (id) TYPE minmax GRANULARITY 3 ) ENGINE=ReplicatedMergeTree() PARTITION BY dt ORDER BY (dt, id) SETTINGS storage_policy='s3' ,min_rows_for_wide_part=8192 on node1 test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[8192-12] Executing query INSERT INTO s3_test VALUES ('2020-01-01',1,'gNrrvLhuPn'),('2020-01-01',2,'FsMlpslLzO'),('2020-01-01',3,'OYjUjvOQBZ'),('2020-01-01',4,'scRUZrOEEi'),('2020-01-01',5,'xbqfJtfqzq'),('2020-01-01',6,'JGjOfPEFRs'),('2020-01-01',7,'yjWanwUUKb'),('2020-01-01',8,'ylitSEQtoG'),('2020-01-01',9,'eOfkgPUigv'),('2020-01-01',10,'JbzZkdILWQ'),('2020-01-01',11,'uFbnwLlvOt'),('2020-01-01',12,'GHostiSoXR'),('2020-01-01',13,'mWOULRQowb'),('2020-01-01',14,'XMiFWtmBKD'),('2020-01-01',15,'oXqKUgfHwv'),('2020-01-01',16,'BSqfNZMzQa'),('2020-01-01',17,'cFjMBwEHeR'),('2020-01-01',18,'WCjdtJLRYS'),('2020-01-01',19,'zJIWwZEUMq'),('2020-01-01',20,'VBYRaZhEPB'),('2020-01-01',21,'XObVGeqMlL'),('2020-01-01',22,'iPjLLFLOFx'),('2020-01-01',23,'yjmYCdXNbp'),('2020-01-01',24,'VpusxGQzNM'),('2020-01-01',25,'poUQBMHaUy'),('2020-01-01',26,'RSUpWStsil'),('2020-01-01',27,'kEUMLeJKOC'),('2020-01-01',28,'ipKHfjePLw'),('2020-01-01',29,'nuMArSfPMg'),('2020-01-01',30,'fidXBmOrWM'),('2020-01-01',31,'xkIZdzoIpJ'),('2020-01-01',32,'KJw on node1 Executing query ALTER TABLE test_rename_with_parallel_insert RENAME COLUMN foo2 to num2 on node1 Executing query SELECT count() FROM test_rename_with_parallel_insert WHERE num2 % 1000 > 0 on node1 thread 0: insert for 2000-01-01: [8, 7, 3, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO s3_test VALUES ('2020-01-02',1,'agRshzCoEs'),('2020-01-02',2,'TefLQHewqG'),('2020-01-02',3,'ZpKyiCdflB'),('2020-01-02',4,'XSqFTgYFpi'),('2020-01-02',5,'ZCulxQtfau'),('2020-01-02',6,'YxaXkrowbT'),('2020-01-02',7,'QLiGvBQqfA'),('2020-01-02',8,'WcJZzPuguL'),('2020-01-02',9,'eooLVsFtVD'),('2020-01-02',10,'NiGQoeJHEg'),('2020-01-02',11,'uhIAzLBxiQ'),('2020-01-02',12,'tYUqaLrMGS'),('2020-01-02',13,'HQhGqGFGve'),('2020-01-02',14,'PCRZYpHFbS'),('2020-01-02',15,'HOJGZUWbWl'),('2020-01-02',16,'ohaxWMcoiq'),('2020-01-02',17,'ogmjkbPcMk'),('2020-01-02',18,'enVyxJHfFw'),('2020-01-02',19,'pXcduPssUU'),('2020-01-02',20,'aYKkanAlcy'),('2020-01-02',21,'xJqkuTKAhp'),('2020-01-02',22,'SocCgIjrVW'),('2020-01-02',23,'dbyGlMwMFo'),('2020-01-02',24,'uckxkvMVVM'),('2020-01-02',25,'GGGdBsWZHX'),('2020-01-02',26,'GzkAhthPde'),('2020-01-02',27,'oQXGHzbwiv'),('2020-01-02',28,'EPKpZtWCXE'),('2020-01-02',29,'xnXfKvFETU'),('2020-01-02',30,'OiMVSzoFbx'),('2020-01-02',31,'hWZcNrtsOK'),('2020-01-02',32,'jIF on node2 Executing query DROP TABLE IF EXISTS limited_sends0 SYNC on node3 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_insert SYNC on node1 thread 1: insert for 2000-01-02: [3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO s3_test VALUES ('2020-01-03',1,'pUZHmVATcU'),('2020-01-03',2,'FXENYGgRvZ'),('2020-01-03',3,'FQqdEgxXKE'),('2020-01-03',4,'kstNgpVULt'),('2020-01-03',5,'zVtUZPwGFo'),('2020-01-03',6,'GtZaepmXKx'),('2020-01-03',7,'lahBHOXvHU'),('2020-01-03',8,'YbysiVrwnC'),('2020-01-03',9,'sBWYjsqPNd'),('2020-01-03',10,'enknTgXOFP'),('2020-01-03',11,'DsUqgjSrOd'),('2020-01-03',12,'wudbNbFVur'),('2020-01-03',13,'iEDYhmHSjA'),('2020-01-03',14,'wwVoluirdj'),('2020-01-03',15,'uWBZczYNLN'),('2020-01-03',16,'cUzorEFbho'),('2020-01-03',17,'uBOHWqyErq'),('2020-01-03',18,'OOodYQTOYX'),('2020-01-03',19,'kTHTzfKFtL'),('2020-01-03',20,'uZwraIlCBO'),('2020-01-03',21,'iFwJoVClGE'),('2020-01-03',22,'zdBRueQEBD'),('2020-01-03',23,'KyoWMhCxtz'),('2020-01-03',24,'appxOktqrB'),('2020-01-03',25,'zObyFeXfjA'),('2020-01-03',26,'gssQtZDXMf'),('2020-01-03',27,'dSQFqYGPdD'),('2020-01-03',28,'aXJTVJbMSR'),('2020-01-03',29,'AFHTJRJKCu'),('2020-01-03',30,'mOmcFZZWxg'),('2020-01-03',31,'zSEEAEjuJX'),('2020-01-03',32,'DZm on node3 Executing query DROP TABLE IF EXISTS limited_sends1 SYNC on node3 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_insert SYNC on node2 Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True thread 0: insert for 2000-01-01: [9, 3, 7, 8, 1, 10, 3, 10, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT * FROM s3_test order by dt, id FORMAT Values on node1 Executing query DROP TABLE IF EXISTS limited_sends2 SYNC on node3 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Closing connection to 172.16.2.2:2281 Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.2.3, port:2281, use_ssl:True Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True Executing query SELECT * FROM s3_test order by dt, id FORMAT Values on node2 Executing query DROP TABLE IF EXISTS limited_sends3 SYNC on node3 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) thread 1: insert for 2000-01-02: [5, 4, 9, 4, 9, 2, 3, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Closing connection to 172.16.2.3:2281 Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.2.4, port:2281, use_ssl:True Connecting to 172.16.2.4(172.16.2.4):2281, use_ssl: True Executing query DROP TABLE IF EXISTS test_rename_with_parallel_insert SYNC on node3 Executing query CREATE TABLE test_remove_policy ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 Executing query SELECT * FROM s3_test order by dt, id FORMAT Values on node3 Executing query SELECT policy_name FROM system.storage_policies on node1 Executing query DROP TABLE IF EXISTS limited_sends4 SYNC on node3 http://172.16.9.7:9001 "GET /root?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=data%2F HTTP/1.1" 200 0 Executing query DROP TABLE IF EXISTS s3_test SYNC on node1 [gw8] PASSED test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[8192-12] Executing query SYSTEM RELOAD CONFIG on node1 Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node1 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_insert SYNC on node4 [gw6] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server Executing query SELECT policy_name FROM system.storage_policies on node1 test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch Executing query CREATE TABLE should_execute_table(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/should_execute_table', '0') ORDER BY tuple() PARTITION BY key SETTINGS max_replicated_fetches_network_bandwidth=3505253 on node1 Executing query DROP TABLE IF EXISTS s3_test SYNC on node2 thread 0: insert for 2000-01-01: [9, 8, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c cat /var/log/clickhouse-server/clickhouse-server.log] Executing query CREATE TABLE should_execute_table(key UInt64, data String) ENGINE = ReplicatedMergeTree('/clickhouse/tables/should_execute_table', '1') ORDER BY tuple() PARTITION BY key SETTINGS max_replicated_fetches_network_bandwidth=3505253 on node2 Stdout:2025.04.02 03:32:24.209818 [ 6846 ] {} SentryWriter: Sending crash reports is disabled Stdout:2025.04.02 03:32:24.256790 [ 6846 ] {} Application: Starting ClickHouse 25.2.2.96963.altinityantalya.96963 (revision: 54496, git hash: 25f7c2a766916d92d37aee82d9defa5f14b86ec8, build id: 4783FC52DD13D5DFB0294BDDD711047195FEB5A6), PID 6846 Stdout:2025.04.02 03:32:24.256912 [ 6846 ] {} Application: starting up Stdout:2025.04.02 03:32:24.256923 [ 6846 ] {} Application: OS name: Linux, version: 5.15.0-130-generic, architecture: x86_64 Stdout:2025.04.02 03:32:24.257056 [ 6846 ] {} Jemalloc: Value for background_thread set to true (from true) Stdout:2025.04.02 03:32:24.261967 [ 6846 ] {} Application: Available RAM: 30.60 GiB; logical cores: 16; used cores: 16. Stdout:2025.04.02 03:32:24.261998 [ 6846 ] {} Application: Available CPU instruction sets: SSE, SSE2, SSE3, SSSE3, SSE41, SSE42, F16C, POPCNT, BMI1, BMI2, PCLMUL, AES, AVX, FMA, AVX2, SHA, ADX, RDRAND, RDSEED, RDTSCP, CLFLUSHOPT, CLWB, XSAVE, OSXSAVE Stdout:2025.04.02 03:32:24.262056 [ 6846 ] {} Pipe: Pipe capacity is 1.00 MiB Stdout:2025.04.02 03:32:24.262799 [ 6846 ] {} CgroupsReader: Will create cgroup reader from '/sys/fs/cgroup/' (cgroups version: v2) Stdout:2025.04.02 03:32:24.262992 [ 6846 ] {} AsynchronousMetrics: Scanning /sys/class/thermal Stdout:2025.04.02 03:32:24.263011 [ 6846 ] {} AsynchronousMetrics: Scanning /sys/block Stdout:2025.04.02 03:32:24.263119 [ 6846 ] {} AsynchronousMetrics: Scanning /sys/devices/system/edac Stdout:2025.04.02 03:32:24.263139 [ 6846 ] {} AsynchronousMetrics: Scanning /sys/class/hwmon Stdout:2025.04.02 03:32:24.263228 [ 6846 ] {} StatusFile: Status file /var/lib/clickhouse/status already exists - unclean restart. Contents: Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node2 Stdout:PID: 6071 Stdout:Started at: 2025-04-02 03:32:19 Stdout:Revision: 54496 Stdout: Stdout:2025.04.02 03:32:24.441549 [ 6846 ] {} Application: Integrity check of the executable successfully passed (checksum: 2D3F861318CFEDFBBB742B17F83CC397) Stdout:2025.04.02 03:32:24.441650 [ 6846 ] {} Application: Will do mlock to prevent executable memory from being paged out. It may take a few seconds. Stdout:2025.04.02 03:32:24.445399 [ 6846 ] {} Application: The memory map of clickhouse executable has been mlock'ed, total 287.17 MiB Stdout:2025.04.02 03:32:24.445431 [ 6846 ] {} MemoryWorker: Starting background memory thread with period of 50ms, using Cgroups as source Stdout:2025.04.02 03:32:24.445507 [ 6846 ] {} BackgroundSchedulePool/BgSchPool: Create BackgroundSchedulePool with 512 threads Stdout:2025.04.02 03:32:24.500866 [ 6879 ] {} MemoryTracker: Correcting the value of global memory tracker from 2.62 MiB to 94.90 MiB Stdout:2025.04.02 03:32:24.515466 [ 7375 ] {} CancellationChecker: Started worker function Stdout:2025.04.02 03:32:24.515489 [ 6846 ] {} Application: rlimit on number of file descriptors is 1048576 Stdout:2025.04.02 03:32:24.515549 [ 6846 ] {} Application: rlimit on number of threads is 18446744073709551615 Stdout:2025.04.02 03:32:24.515568 [ 6846 ] {} Application: Initializing DateLUT. Stdout:2025.04.02 03:32:24.515573 [ 6846 ] {} Application: Initialized DateLUT with time zone 'Etc/UTC'. Stdout:2025.04.02 03:32:24.515608 [ 6846 ] {} Context: Setting up /var/lib/clickhouse/tmp/ to store temporary data in it Stdout:2025.04.02 03:32:24.515802 [ 6846 ] {} Application: Configuration parameter 'interserver_http_host' doesn't exist or exists and empty. Will use 'node1' as replica host. Stdout:2025.04.02 03:32:24.515818 [ 6846 ] {} Application: Initializing interserver credentials. Stdout:2025.04.02 03:32:24.515932 [ 6846 ] {} NamedCollectionsMetadataStorage: Using local storage for named collections at path: /var/lib/clickhouse/named_collections Stdout:2025.04.02 03:32:24.515954 [ 6846 ] {} FileCacheFactory: Will load 0 caches from default cache config Stdout:2025.04.02 03:32:24.517649 [ 6846 ] {} ConfigReloader: Loading config '/etc/clickhouse-server/config.xml' Stdout:2025.04.02 03:32:24.517660 [ 6846 ] {} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/config.xml'. Stdout:2025.04.02 03:32:24.518502 [ 6846 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:24.518585 [ 6846 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:24.518665 [ 6846 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_enable_keeper_async_replication.xml'. Stdout:2025.04.02 03:32:24.518719 [ 6846 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_instance_config.xml'. Stdout:2025.04.02 03:32:24.518870 [ 6846 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/logs_config.xml'. Stdout:2025.04.02 03:32:24.518949 [ 6846 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/storage_configuration.xml'. Stdout:2025.04.02 03:32:24.523570 [ 6846 ] {} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'. Stdout:2025.04.02 03:32:24.523668 [ 6846 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performing update on configuration Stdout:2025.04.02 03:32:24.529945 [ 6846 ] {} Application: Changed setting 'max_server_memory_usage' to 27.54 GiB (30.60 GiB available memory * 0.90 max_server_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:24.529958 [ 6846 ] {} Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.30 GiB (30.60 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:24.529966 [ 6846 ] {} Application: Merges and mutations memory limit is set to 15.30 GiB Stdout:2025.04.02 03:32:24.530083 [ 6846 ] {} Application: Setting max_remote_read_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:24.530087 [ 6846 ] {} Application: Setting max_remote_write_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:24.530094 [ 6846 ] {} Application: ConcurrencyControl limit is set to 32 CPU slots with 'fair_round_robin' scheduler Stdout:2025.04.02 03:32:24.530104 [ 6846 ] {} BackgroundSchedulePool/BgBufSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:24.531447 [ 6846 ] {} BackgroundSchedulePool/BgMBSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:24.533077 [ 6846 ] {} BackgroundSchedulePool/BgDistSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:24.534882 [ 6846 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration Stdout:2025.04.02 03:32:24.534922 [ 6846 ] {} ConfigReloader: Config reload interval set to 2000ms Stdout:2025.04.02 03:32:24.535881 [ 6846 ] {} Application: Listening for replica communication (interserver): http://0.0.0.0:9009 Stdout:2025.04.02 03:32:24.538092 [ 6846 ] {} ConfigReloader: Loading config '/etc/clickhouse-server/users.xml' Stdout:2025.04.02 03:32:24.538109 [ 6846 ] {} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/users.xml'. Stdout:2025.04.02 03:32:24.538326 [ 6846 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:24.538397 [ 6846 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:24.538481 [ 6846 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/users.d/0_common_instance_users.xml'. Stdout:2025.04.02 03:32:24.538544 [ 6846 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/users.d/0_random_settings.xml'. Stdout:2025.04.02 03:32:24.539416 [ 6846 ] {} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'. Stdout:2025.04.02 03:32:24.539448 [ 6846 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performing update on configuration Stdout:2025.04.02 03:32:24.540498 [ 6846 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performed update on configuration Stdout:2025.04.02 03:32:24.540518 [ 6846 ] {} ConfigReloader: Config reload interval set to 2000ms Stdout:2025.04.02 03:32:24.541002 [ 6846 ] {} Access(user directories): Added users_xml access storage 'users_xml', path: /etc/clickhouse-server/users.xml Stdout:2025.04.02 03:32:24.541191 [ 6846 ] {} Access(user directories): Added local_directory access storage 'local_directory', path: /var/lib/clickhouse/access/ Stdout:2025.04.02 03:32:24.541303 [ 6846 ] {} CgroupsMemoryUsageObserver: Started cgroup current memory usage observer thread Stdout:2025.04.02 03:32:24.541565 [ 7429 ] {} CgroupsMemoryUsageObserver: Memory amount initially available to the process is 30.60 GiB Stdout:2025.04.02 03:32:24.545885 [ 6846 ] {} Context: Initialized background executor for merges and mutations with num_threads=16, num_tasks=32, scheduling_policy=round_robin Stdout:2025.04.02 03:32:24.546631 [ 6846 ] {} Context: Initialized background executor for move operations with num_threads=8, num_tasks=8 Stdout:2025.04.02 03:32:24.548167 [ 6846 ] {} Context: Initialized background executor for fetches with num_threads=16, num_tasks=16 Stdout:2025.04.02 03:32:24.548947 [ 6846 ] {} Context: Initialized background executor for common operations (e.g. clearing old parts) with num_threads=8, num_tasks=8 Stdout:2025.04.02 03:32:24.550932 [ 6846 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. Stdout:2025.04.02 03:32:24.552152 [ 6846 ] {} DNSCacheUpdater: Update period 15 seconds Stdout:2025.04.02 03:32:24.552194 [ 6864 ] {} DNSResolver: Updating DNS cache Stdout:2025.04.02 03:32:24.552208 [ 6864 ] {} DNSResolver: Updated DNS cache Stdout:2025.04.02 03:32:24.552228 [ 6846 ] {} Application: Loading metadata from /var/lib/clickhouse/ Stdout:2025.04.02 03:32:24.555437 [ 6846 ] {} Context: Database disk name: default Stdout:2025.04.02 03:32:24.555452 [ 6846 ] {} Context: Database disk name: default, path: /var/lib/clickhouse/ Stdout:2025.04.02 03:32:24.555477 [ 6846 ] {} loadSystemDatabase: metadata_file_path metadata/system.sql, existsFile true Stdout:2025.04.02 03:32:24.555558 [ 6846 ] {} loadSystemDatabase: metadata_file_path metadata/information_schema.sql, existsFile true Stdout:2025.04.02 03:32:24.555595 [ 6846 ] {} loadSystemDatabase: metadata_file_path metadata/INFORMATION_SCHEMA.sql, existsFile true Stdout:2025.04.02 03:32:24.555671 [ 6846 ] {} DatabaseAtomic (system): Metadata processed, database system has 0 tables and 0 dictionaries in total. Stdout:2025.04.02 03:32:24.555680 [ 6846 ] {} TablesLoader: Parsed metadata of 0 tables in 1 databases in 5.4522e-05 sec Stdout:2025.04.02 03:32:24.555688 [ 6846 ] {} ReferentialDeps: No tables Stdout:2025.04.02 03:32:24.555691 [ 6846 ] {} LoadingDeps: No tables Stdout:2025.04.02 03:32:24.555733 [ 6846 ] {} AsyncLoader: Prioritize load job 'startup Atomic database system': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:24.555738 [ 6846 ] {} AsyncLoader: Prioritize load job 'startup Ordinary database system': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:24.555744 [ 6846 ] {} AsyncLoader: Schedule load job 'startup Ordinary database system' into ForegroundLoad Stdout:2025.04.02 03:32:24.555748 [ 6846 ] {} AsyncLoader: Schedule load job 'startup Atomic database system' into ForegroundLoad Stdout:2025.04.02 03:32:24.555753 [ 6846 ] {} AsyncLoader: Change current priority: none -> 0 Stdout:2025.04.02 03:32:24.555757 [ 6846 ] {} AsyncLoader: Spawn loader worker #1 in ForegroundLoad Stdout:2025.04.02 03:32:24.555823 [ 6846 ] {} AsyncLoader: Wait load job 'startup Atomic database system' in ForegroundLoad Stdout:2025.04.02 03:32:24.555887 [ 7496 ] {} AsyncLoader: Execute load job 'startup Ordinary database system' in ForegroundLoad Stdout:2025.04.02 03:32:24.555924 [ 7496 ] {} AsyncLoader: Finish load job 'startup Ordinary database system' with status OK Stdout:2025.04.02 03:32:24.555934 [ 7496 ] {} AsyncLoader: Spawn loader worker #2 in ForegroundLoad Stdout:2025.04.02 03:32:24.556018 [ 7496 ] {} AsyncLoader: Execute load job 'startup Atomic database system' in ForegroundLoad Stdout:2025.04.02 03:32:24.556063 [ 7496 ] {} AsyncLoader: Finish load job 'startup Atomic database system' with status OK Stdout:2025.04.02 03:32:24.556069 [ 7496 ] {} AsyncLoader: Stop worker in ForegroundLoad Stdout:2025.04.02 03:32:24.556098 [ 6846 ] {} SystemLog: Creating system.query_log from query_log Stdout:2025.04.02 03:32:24.556112 [ 7497 ] {} AsyncLoader: Stop worker in ForegroundLoad Stdout:2025.04.02 03:32:24.556146 [ 7497 ] {} AsyncLoader: Change current priority: 0 -> none Stdout:2025.04.02 03:32:24.565015 [ 6846 ] {} SystemLog: Creating system.query_thread_log from query_thread_log Stdout:2025.04.02 03:32:24.565499 [ 6846 ] {} SystemLog: Creating system.part_log from part_log Stdout:2025.04.02 03:32:24.565830 [ 6846 ] {} SystemLog: Creating system.trace_log from trace_log Stdout:2025.04.02 03:32:24.566126 [ 6846 ] {} SystemLog: Creating system.crash_log from crash_log Stdout:2025.04.02 03:32:24.566278 [ 6846 ] {} SystemLog: Creating system.text_log from text_log Stdout:2025.04.02 03:32:24.566506 [ 6846 ] {} SystemLog: Creating system.metric_log from metric_log Stdout:2025.04.02 03:32:24.571553 [ 6846 ] {} SystemLog: Creating system.latency_log from latency_log Stdout:2025.04.02 03:32:24.571921 [ 6846 ] {} SystemLog: Creating system.error_log from error_log Stdout:2025.04.02 03:32:24.572139 [ 6846 ] {} SystemLog: Not creating system.filesystem_cache_log since corresponding section 'filesystem_cache_log' is missing from config Stdout:2025.04.02 03:32:24.572154 [ 6846 ] {} SystemLog: Not creating system.filesystem_read_prefetches_log since corresponding section 'filesystem_read_prefetches_log' is missing from config Stdout:2025.04.02 03:32:24.572164 [ 6846 ] {} SystemLog: Creating system.s3queue_log from s3queue_log Stdout:2025.04.02 03:32:24.572352 [ 6846 ] {} SystemLog: Not creating system.azure_queue_log since corresponding section 'azure_queue_log' is missing from config Stdout:2025.04.02 03:32:24.572366 [ 6846 ] {} SystemLog: Creating system.asynchronous_metric_log from asynchronous_metric_log Stdout:2025.04.02 03:32:24.572545 [ 6846 ] {} SystemLog: Creating system.opentelemetry_span_log from opentelemetry_span_log Stdout:2025.04.02 03:32:24.572993 [ 6846 ] {} SystemLog: Creating system.query_views_log from query_views_log Stdout:2025.04.02 03:32:24.573277 [ 6846 ] {} SystemLog: Not creating system.zookeeper_log since corresponding section 'zookeeper_log' is missing from config Stdout:2025.04.02 03:32:24.573293 [ 6846 ] {} SystemLog: Not creating system.session_log since corresponding section 'session_log' is missing from config Stdout:2025.04.02 03:32:24.573302 [ 6846 ] {} SystemLog: Not creating system.transactions_info_log since corresponding section 'transactions_info_log' is missing from config Stdout:2025.04.02 03:32:24.573311 [ 6846 ] {} SystemLog: Creating system.processors_profile_log from processors_profile_log Stdout:2025.04.02 03:32:24.573624 [ 6846 ] {} SystemLog: Creating system.asynchronous_insert_log from asynchronous_insert_log Stdout:2025.04.02 03:32:24.573867 [ 6846 ] {} SystemLog: Creating system.backup_log from backup_log Stdout:2025.04.02 03:32:24.574176 [ 6846 ] {} SystemLog: Creating system.blob_storage_log from blob_storage_log Stdout:2025.04.02 03:32:24.574532 [ 6846 ] {} SystemLog: Creating system.query_metric_log from query_metric_log Stdout:2025.04.02 03:32:24.579414 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.579430 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.580837 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.580853 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.580871 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.580875 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.580906 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.580912 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.580943 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.580950 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581023 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581033 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581169 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581175 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581250 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581257 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581300 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581304 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581347 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581351 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581449 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581455 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581491 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581495 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581523 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581527 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581557 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581561 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581594 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581598 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581633 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581637 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581657 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581661 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581681 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581685 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581710 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581822 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581847 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581850 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581868 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581871 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581890 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581894 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581909 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581912 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581935 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581941 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.581961 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.581965 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582013 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582018 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582056 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582060 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582184 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582191 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582218 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582227 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582247 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582250 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582267 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582270 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582295 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582298 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582336 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582341 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582375 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582381 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582430 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582446 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582496 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582500 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582565 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582569 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582637 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582640 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582672 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582682 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582938 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582945 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582973 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582976 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.582993 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.582996 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583018 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583021 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583067 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583074 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583096 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583099 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583115 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583118 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583158 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583162 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583192 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583195 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583223 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583227 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583358 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583363 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583462 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583474 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583508 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583512 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583530 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583534 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583575 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583579 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583619 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583623 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583643 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583647 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583662 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583666 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583780 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583785 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583915 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583920 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.583975 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.583979 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584086 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584092 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584184 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584188 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584233 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584236 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584273 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584277 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584411 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584417 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584480 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584485 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584525 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584529 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584564 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584568 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584648 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584658 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584835 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584851 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.584948 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.584958 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585034 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585052 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585087 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585091 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585148 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585153 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585186 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585190 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585241 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585245 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585278 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585282 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585300 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585303 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585333 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585337 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585367 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585371 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585398 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585401 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585449 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585457 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585511 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585518 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585551 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585559 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585585 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585589 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585621 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585625 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585650 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585653 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585697 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585704 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585772 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585779 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585823 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585832 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585884 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585893 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585929 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585933 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585957 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585964 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.585987 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.585991 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.586009 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.586012 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.586060 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.586064 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.586088 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.586092 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.586114 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.586120 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.586154 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.586158 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.586193 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.586197 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.586264 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.586268 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.593774 [ 6846 ] {} DatabaseCatalog: Found 8 partially dropped tables. Will load them and retry removal. Stdout:2025.04.02 03:32:24.594238 [ 7513 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f) from metadata_dropped/default.test_add_disk_to_policy.e1af3e60-fe6e-4895-a02b-fced2189743f.sql Stdout:2025.04.02 03:32:24.594525 [ 7513 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:24.594988 [ 7513 ] {} StoragePolicy (small_jbod_with_external): Storage policy small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:24.594999 [ 7513 ] {} StoragePolicySelector: Storage policy `small_jbod_with_external` loaded Stdout:2025.04.02 03:32:24.595131 [ 7513 ] {} StoragePolicy (one_more_small_jbod_with_external): Storage policy one_more_small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:24.595137 [ 7513 ] {} StoragePolicySelector: Storage policy `one_more_small_jbod_with_external` loaded Stdout:2025.04.02 03:32:24.595265 [ 7513 ] {} StoragePolicy (jbods_with_external): Storage policy jbods_with_external created, total volumes 2 Stdout:2025.04.02 03:32:24.595270 [ 7513 ] {} StoragePolicySelector: Storage policy `jbods_with_external` loaded Stdout:2025.04.02 03:32:24.595386 [ 7513 ] {} StoragePolicy (moving_jbod_with_external): Storage policy moving_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:24.595393 [ 7513 ] {} StoragePolicySelector: Storage policy `moving_jbod_with_external` loaded Stdout:2025.04.02 03:32:24.595455 [ 7513 ] {} StorageConfiguration: Volume `small` max_data_part_size is too low (2.00 MiB < 8.00 MiB) Stdout:2025.04.02 03:32:24.595535 [ 7513 ] {} StoragePolicy (default_disk_with_external): Storage policy default_disk_with_external created, total volumes 2 Stdout:2025.04.02 03:32:24.595543 [ 7513 ] {} StoragePolicySelector: Storage policy `default_disk_with_external` loaded Stdout:2025.04.02 03:32:24.595702 [ 7513 ] {} StorageConfiguration: Volume `special_warning_small_volume` max_data_part_size is too low (1.00 KiB < 8.00 MiB) Stdout:2025.04.02 03:32:24.595782 [ 7513 ] {} StoragePolicy (special_warning_policy): Storage policy special_warning_policy created, total volumes 4 Stdout:2025.04.02 03:32:24.595786 [ 7513 ] {} StoragePolicySelector: Storage policy `special_warning_policy` loaded Stdout:2025.04.02 03:32:24.595862 [ 7513 ] {} StoragePolicy (remove_policy_cool_policy): Storage policy remove_policy_cool_policy created, total volumes 1 Stdout:2025.04.02 03:32:24.595866 [ 7513 ] {} StoragePolicySelector: Storage policy `remove_policy_cool_policy` loaded Stdout:2025.04.02 03:32:24.595879 [ 7513 ] {} StoragePolicy (default): Storage policy default created, total volumes 1 Stdout:2025.04.02 03:32:24.595971 [ 7520 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c) from metadata_dropped/default.test_add_policy.d69b6b75-0af1-4149-a0f9-c13bb05ea67c.sql Stdout:2025.04.02 03:32:24.596196 [ 7520 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:24.594149 [ 7518 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3) from metadata_dropped/default.test_add_disk.8a487b06-210e-4c1c-91cf-235fb0fcffb3.sql Stdout:2025.04.02 03:32:24.594174 [ 7519 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174) from metadata_dropped/default.test_add_disk.ab1b9e7f-3e87-42e0-987c-542af287b174.sql Stdout:2025.04.02 03:32:24.596665 [ 7513 ] {} default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f): Loading data parts Stdout:2025.04.02 03:32:24.597328 [ 7520 ] {} default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c): Loading data parts Stdout:2025.04.02 03:32:24.598239 [ 7525 ] {} DatabaseCatalog: Trying load partially dropped table default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce) from metadata_dropped/default.test_remove_disk.b18b3533-cce9-430c-ba4b-cbd718231cce.sql Stdout:2025.04.02 03:32:24.598430 [ 7518 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:24.598519 [ 7519 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:24.598628 [ 7522 ] {} DatabaseCatalog: Trying load partially dropped table default.test_new_policy_works (9288e96b-23dc-4115-80b9-90a084e01018) from metadata_dropped/default.test_new_policy_works.9288e96b-23dc-4115-80b9-90a084e01018.sql Stdout:2025.04.02 03:32:24.603222 [ 7518 ] {} default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3): Loading data parts Stdout:2025.04.02 03:32:24.608806 [ 7521 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5) from metadata_dropped/default.test_add_volume_to_policy.b03aef8f-055b-454d-bac3-7055c6fd12b5.sql Stdout:2025.04.02 03:32:24.610798 [ 7513 ] {} default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f): There are no data parts Stdout:2025.04.02 03:32:24.610841 [ 7518 ] {} default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3): There are no data parts Stdout:2025.04.02 03:32:24.611014 [ 7526 ] {} DatabaseCatalog: Trying load partially dropped table default.test_remove_disk_from_policy (ee9c3908-42fe-4bec-94ec-e0eae1e532db) from metadata_dropped/default.test_remove_disk_from_policy.ee9c3908-42fe-4bec-94ec-e0eae1e532db.sql Stdout:2025.04.02 03:32:24.611053 [ 7525 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:24.611095 [ 7520 ] {} default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c): There are no data parts Stdout:2025.04.02 03:32:24.611425 [ 7526 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:24.611553 [ 7519 ] {} default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174): Loading data parts Stdout:2025.04.02 03:32:24.611711 [ 7522 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:24.611770 [ 7519 ] {} default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174): There are no data parts Stdout:2025.04.02 03:32:24.611953 [ 7521 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:24.612560 [ 7522 ] {} DatabaseCatalog: Cannot load partially dropped table default.test_new_policy_works (9288e96b-23dc-4115-80b9-90a084e01018) from: metadata_dropped/default.test_new_policy_works.9288e96b-23dc-4115-80b9-90a084e01018.sql. Parsed query: ATTACH TABLE default.test_new_policy_works UUID '9288e96b-23dc-4115-80b9-90a084e01018' (`d` UInt64) ENGINE = MergeTree ORDER BY d SETTINGS storage_policy = 'cool_policy', index_granularity = 8192. Will remove metadata and store/928/9288e96b-23dc-4115-80b9-90a084e01018/. Garbage may be left in ZooKeeper.: Code: 478. DB::Exception: Unknown storage policy `cool_policy`. (UNKNOWN_POLICY), Stack trace (when copying this message, always include the lines below): Stdout: Stdout:0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000cfb8fbb Stdout:1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007ff830c Stdout:2. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x0000000007ff802b Stdout:3. DB::StoragePolicySelector::get(String const&) const @ 0x00000000105c18e1 Stdout:4. DB::Context::getStoragePolicy(String const&) const @ 0x0000000010b34cbf Stdout:5. DB::MergeTreeData::getStoragePolicy() const @ 0x000000001201df88 Stdout:6. DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x00000000120136a8 Stdout:7. DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x00000000123e6548 Stdout:8. DB::create(DB::StorageFactory::Arguments const&) @ 0x00000000123e2c8c Stdout:9. DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x00000000119e37bf Stdout:10. DB::createTableFromAST(DB::ASTCreateQuery, String const&, String const&, std::shared_ptr, DB::LoadingStrictnessLevel) @ 0x00000000104454aa Stdout:11. DB::DatabaseCatalog::enqueueDroppedTableCleanup(DB::StorageID, std::shared_ptr, String, bool) @ 0x0000000010c038a9 Stdout:12. void std::__function::__policy_invoker::__call_impl[abi:ne190107]>(std::__function::__policy_storage const*) @ 0x0000000010c14b0e Stdout:13. void DB::ThreadPoolCallbackRunnerLocal>, std::function>::executeCallback>(std::promise&, std::function&&) @ 0x000000000efb3e95 Stdout:14. DB::ThreadPoolCallbackRunnerLocal>, std::function>::operator()(std::function&&, Priority, std::optional)::'lambda'()::operator()() @ 0x000000000efb3c9b Stdout:15. ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x000000000d0992db Stdout:16. void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl>::ThreadFromThreadPool::*)(), ThreadPoolImpl>::ThreadFromThreadPool*>(void (ThreadPoolImpl>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl>::ThreadFromThreadPool*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000d09fda3 Stdout:17. ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000000d09656f Stdout:18. void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000000d09d85a Stdout:19. ? @ 0x00007f3c56d4dac3 Stdout:20. ? @ 0x00007f3c56ddf850 Stdout: (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:24.612868 [ 7525 ] {} default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce): Loading data parts Stdout:2025.04.02 03:32:24.612871 [ 7526 ] {} default.test_remove_disk_from_policy (ee9c3908-42fe-4bec-94ec-e0eae1e532db): Loading data parts Stdout:2025.04.02 03:32:24.613214 [ 7526 ] {} default.test_remove_disk_from_policy (ee9c3908-42fe-4bec-94ec-e0eae1e532db): There are no data parts Stdout:2025.04.02 03:32:24.613241 [ 7525 ] {} default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce): There are no data parts Stdout:2025.04.02 03:32:24.616699 [ 7521 ] {} default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5): Loading data parts Stdout:2025.04.02 03:32:24.624017 [ 7521 ] {} default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5): There are no data parts Stdout:2025.04.02 03:32:24.624365 [ 6846 ] {} DatabaseAtomic (default): Metadata processed, database default has 0 tables and 0 dictionaries in total. Stdout:2025.04.02 03:32:24.624381 [ 6846 ] {} TablesLoader: Parsed metadata of 0 tables in 1 databases in 4.9622e-05 sec Stdout:2025.04.02 03:32:24.624386 [ 6846 ] {} ReferentialDeps: No tables Stdout:2025.04.02 03:32:24.624399 [ 6846 ] {} LoadingDeps: No tables Stdout:2025.04.02 03:32:24.624433 [ 6846 ] {} loadMetadata: Start asynchronous loading of databases Stdout:2025.04.02 03:32:24.624440 [ 6846 ] {} AsyncLoader: Schedule load job 'startup Ordinary database default' into BackgrndStartup Stdout:2025.04.02 03:32:24.624444 [ 6846 ] {} AsyncLoader: Schedule load job 'startup Atomic database default' into BackgrndStartup Stdout:2025.04.02 03:32:24.624448 [ 6846 ] {} AsyncLoader: Change current priority: none -> 2 Stdout:2025.04.02 03:32:24.624452 [ 6846 ] {} AsyncLoader: Spawn loader worker #1 in BackgrndStartup Stdout:2025.04.02 03:32:24.624501 [ 6846 ] {} UserDefinedSQLObjectsLoaderFromDisk: Loading user defined objects from /var/lib/clickhouse/user_defined/ Stdout:2025.04.02 03:32:24.624509 [ 6846 ] {} UserDefinedSQLObjectsLoaderFromDisk: The directory for user defined objects (/var/lib/clickhouse/user_defined/) does not exist: nothing to load Stdout:2025.04.02 03:32:24.624661 [ 6846 ] {} WorkloadEntityDiskStorage: Loading workload entities from /var/lib/clickhouse/workload/ Stdout:2025.04.02 03:32:24.624674 [ 6846 ] {} WorkloadEntityDiskStorage: The directory for workload entities (/var/lib/clickhouse/workload/) does not exist: nothing to load Stdout:2025.04.02 03:32:24.624700 [ 7519 ] {} AsyncLoader: Execute load job 'startup Ordinary database default' in BackgrndStartup Stdout:2025.04.02 03:32:24.624719 [ 7519 ] {} AsyncLoader: Finish load job 'startup Ordinary database default' with status OK Stdout:2025.04.02 03:32:24.624726 [ 7519 ] {} AsyncLoader: Spawn loader worker #2 in BackgrndStartup Stdout:2025.04.02 03:32:24.624743 [ 7519 ] {} AsyncLoader: Execute load job 'startup Atomic database default' in BackgrndStartup Stdout:2025.04.02 03:32:24.624758 [ 7519 ] {} AsyncLoader: Finish load job 'startup Atomic database default' with status OK Stdout:2025.04.02 03:32:24.624764 [ 7519 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:24.624786 [ 7522 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:24.624800 [ 7522 ] {} AsyncLoader: Change current priority: 2 -> none Stdout:2025.04.02 03:32:24.633630 [ 6846 ] {} ZooKeeperClient: Adding ZooKeeper host zoo2:2181 (172.16.8.4:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:24.633876 [ 6846 ] {} ZooKeeperClient: Adding ZooKeeper host zoo3:2181 (172.16.8.3:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:24.634125 [ 6846 ] {} ZooKeeperClient: Adding ZooKeeper host zoo1:2181 (172.16.8.2:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:24.636383 [ 6846 ] {} ZooKeeperClient: Connected to ZooKeeper at 172.16.8.4:2181 with session_id 14 Stdout:2025.04.02 03:32:24.636585 [ 6846 ] {} ZooKeeperClient: Keeper feature flag FILTERED_LIST: enabled Stdout:2025.04.02 03:32:24.636602 [ 6846 ] {} ZooKeeperClient: Keeper feature flag MULTI_READ: disabled Stdout:2025.04.02 03:32:24.636606 [ 6846 ] {} ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: enabled Stdout:2025.04.02 03:32:24.636610 [ 6846 ] {} ZooKeeperClient: Keeper feature flag CREATE_IF_NOT_EXISTS: disabled Stdout:2025.04.02 03:32:24.636616 [ 6846 ] {} ZooKeeperClient: Keeper feature flag REMOVE_RECURSIVE: enabled Stdout:2025.04.02 03:32:24.636640 [ 6846 ] {} ZooKeeper: Initialized, hosts: zoo1:2181,zoo2:2181,zoo3:2181 Stdout:2025.04.02 03:32:24.638070 [ 6846 ] {} Application: Loaded metadata. Stdout:2025.04.02 03:32:24.638723 [ 6846 ] {} bool DB::(anonymous namespace)::checkPermissionsImpl(): Code: 412. DB::Exception: Can't receive Netlink response: error -2. (NETLINK_ERROR) (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:24.638755 [ 6846 ] {} Application: Tasks stats provider: procfs Stdout:2025.04.02 03:32:24.638842 [ 6846 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:24.638848 [ 6846 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:24.639263 [ 6846 ] {} MySQLHandlerFactory: Failed to create SSL context. SSL will be disabled. Error: Poco::Exception. Code: 1000, e.code() = 0, SSL Exception: Configuration error: no certificate file has been specified (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:24.639303 [ 6846 ] {} MySQLHandlerFactory: Failed to read RSA key pair from server certificate. Error: Code: 139. DB::Exception: Certificate file is not set. (NO_ELEMENTS_IN_CONFIG) (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:24.639308 [ 6846 ] {} MySQLHandlerFactory: Generating new RSA key pair. Stdout:2025.04.02 03:32:24.716260 [ 6846 ] {} CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. Stdout:2025.04.02 03:32:24.716283 [ 6846 ] {} CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. Stdout:2025.04.02 03:32:24.719180 [ 6846 ] {} AsyncLoader: Schedule load job 'startup ddl worker' into BackgrndStartup Stdout:2025.04.02 03:32:24.719200 [ 6846 ] {} AsyncLoader: Change current priority: none -> 2 Stdout:2025.04.02 03:32:24.719205 [ 6846 ] {} AsyncLoader: Spawn loader worker #1 in BackgrndStartup Stdout:2025.04.02 03:32:24.719279 [ 6846 ] {} Application: Listening for http://0.0.0.0:8123 Stdout:2025.04.02 03:32:24.719335 [ 6846 ] {} Application: Listening for native protocol (tcp): 0.0.0.0:9000 Stdout:2025.04.02 03:32:24.719374 [ 6846 ] {} Application: Listening for MySQL compatibility protocol: 0.0.0.0:9004 Stdout:2025.04.02 03:32:24.719410 [ 6846 ] {} Application: Listening for PostgreSQL compatibility protocol: 0.0.0.0:9005 Stdout:2025.04.02 03:32:24.719422 [ 6846 ] {} Application: Ready for connections. Stdout:2025.04.02 03:32:24.719499 [ 7525 ] {} AsyncLoader: Execute load job 'startup ddl worker' in BackgrndStartup Stdout:2025.04.02 03:32:24.719526 [ 7524 ] {} DDLWorker: Starting DDLWorker thread Stdout:2025.04.02 03:32:24.719540 [ 7524 ] {} DDLWorker: Initializing DDLWorker thread Stdout:2025.04.02 03:32:24.719556 [ 7513 ] {} DDLWorker: Started DDLWorker cleanup thread Stdout:2025.04.02 03:32:24.719730 [ 7525 ] {} AsyncLoader: Finish load job 'startup ddl worker' with status OK Stdout:2025.04.02 03:32:24.719739 [ 7525 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:24.719743 [ 7525 ] {} AsyncLoader: Change current priority: 2 -> none Stdout:2025.04.02 03:32:24.722430 [ 7524 ] {} DDLWorker: Initialized DDLWorker thread Stdout:2025.04.02 03:32:24.722672 [ 7513 ] {} DDLWorker: Cleaning queue Stdout:2025.04.02 03:32:24.722812 [ 7524 ] {} DDLWorker: Scheduling tasks Stdout:2025.04.02 03:32:24.722819 [ 7524 ] {} DDLWorker: Don't have unfinished tasks after restarting Stdout:2025.04.02 03:32:24.723191 [ 7524 ] {} DDLWorker: scheduleTasks: initialized=true, size_before_filtering=0, queue_size=0, entries=none..none, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=none Stdout:2025.04.02 03:32:24.723200 [ 7524 ] {} DDLWorker: No tasks to schedule Stdout:2025.04.02 03:32:24.723206 [ 7524 ] {} DDLWorker: Waiting for queue updates Stdout:2025.04.02 03:32:25.328165 [ 6859 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63394 Stdout:2025.04.02 03:32:25.328276 [ 6859 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:25.328302 [ 6859 ] {} TCP-Session-ccd4fe31-4159-430c-a8ed-442fa75d8e6c: Authenticating user 'default' from 172.16.8.1:63394 Stdout:2025.04.02 03:32:25.328349 [ 6859 ] {} TCP-Session-ccd4fe31-4159-430c-a8ed-442fa75d8e6c: ccd4fe31-4159-430c-a8ed-442fa75d8e6c Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:25.328359 [ 6859 ] {} TCP-Session-ccd4fe31-4159-430c-a8ed-442fa75d8e6c: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:25.328498 [ 6859 ] {} ContextAccess (default): Settings: readonly = 0, allow_ddl = true, allow_introspection_functions = false Stdout:2025.04.02 03:32:25.328526 [ 6859 ] {} ContextAccess (default): List of all grants: GRANT CHECK, SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, ROLE ADMIN, CREATE ROW POLICY, ALTER ROW POLICY, DROP ROW POLICY, CREATE QUOTA, ALTER QUOTA, DROP QUOTA, CREATE SETTINGS PROFILE, ALTER SETTINGS PROFILE, DROP SETTINGS PROFILE, ALLOW SQL SECURITY NONE, SHOW ACCESS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* WITH GRANT OPTION, GRANT TABLE ENGINE ON * WITH GRANT OPTION, GRANT CREATE USER, ALTER USER, DROP USER, CREATE ROLE, ALTER ROLE, DROP ROLE, SET DEFINER ON * WITH GRANT OPTION, GRANT ALTER NAMED COLLECTION, CREATE NAMED COLLECTION, DROP NAMED COLLECTION, SHOW NAMED COLLECTIONS, NAMED COLLECTION ON * WITH GRANT OPTION Stdout:2025.04.02 03:32:25.328537 [ 6859 ] {} ContextAccess (default): List of all grants including implicit: GRANT CHECK, SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, ROLE ADMIN, CREATE ROW POLICY, ALTER ROW POLICY, DROP ROW POLICY, CREATE QUOTA, ALTER QUOTA, DROP QUOTA, CREATE SETTINGS PROFILE, ALTER SETTINGS PROFILE, DROP SETTINGS PROFILE, ALLOW SQL SECURITY NONE, SHOW ACCESS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* WITH GRANT OPTION, GRANT TABLE ENGINE ON * WITH GRANT OPTION, GRANT CREATE USER, ALTER USER, DROP USER, CREATE ROLE, ALTER ROLE, DROP ROLE, SET DEFINER ON * WITH GRANT OPTION, GRANT ALTER NAMED COLLECTION, CREATE NAMED COLLECTION, DROP NAMED COLLECTION, SHOW NAMED COLLECTIONS, NAMED COLLECTION ON * WITH GRANT OPTION Stdout:2025.04.02 03:32:25.338513 [ 6859 ] {} TCP-Session-ccd4fe31-4159-430c-a8ed-442fa75d8e6c: ccd4fe31-4159-430c-a8ed-442fa75d8e6c Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:25.338745 [ 6859 ] {7193a536-0a9d-4ac2-9bc3-0e7f01943dcc} executeQuery: (from 172.16.8.1:63394) (query 1, line 1) select 20 (stage: Complete) Stdout:2025.04.02 03:32:25.338913 [ 6859 ] {7193a536-0a9d-4ac2-9bc3-0e7f01943dcc} Planner: Query to stage Complete Stdout:2025.04.02 03:32:25.338981 [ 6859 ] {7193a536-0a9d-4ac2-9bc3-0e7f01943dcc} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:25.339566 [ 6859 ] {7193a536-0a9d-4ac2-9bc3-0e7f01943dcc} executeQuery: Read 1 rows, 1.00 B in 0.000828 sec., 1207.729468599034 rows/sec., 1.18 KiB/sec. Stdout:2025.04.02 03:32:25.339641 [ 6859 ] {7193a536-0a9d-4ac2-9bc3-0e7f01943dcc} TCPHandler: Processed in 0.001323472 sec. Stdout:2025.04.02 03:32:25.349391 [ 6859 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:25.349413 [ 6859 ] {} TCP-Session-ccd4fe31-4159-430c-a8ed-442fa75d8e6c: ccd4fe31-4159-430c-a8ed-442fa75d8e6c Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.420885 [ 6859 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63404 Stdout:2025.04.02 03:32:27.420945 [ 6859 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:27.420959 [ 6859 ] {} TCP-Session-8d0b82d0-dfc8-46b9-a41c-04f38826417a: Authenticating user 'default' from 172.16.8.1:63404 Stdout:2025.04.02 03:32:27.420993 [ 6859 ] {} TCP-Session-8d0b82d0-dfc8-46b9-a41c-04f38826417a: 8d0b82d0-dfc8-46b9-a41c-04f38826417a Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.420999 [ 6859 ] {} TCP-Session-8d0b82d0-dfc8-46b9-a41c-04f38826417a: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.430802 [ 6859 ] {} TCP-Session-8d0b82d0-dfc8-46b9-a41c-04f38826417a: 8d0b82d0-dfc8-46b9-a41c-04f38826417a Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:27.431021 [ 6859 ] {9d88c6e7-712e-453d-9129-09ee87d76050} executeQuery: (from 172.16.8.1:63404) (query 1, line 2) CREATE TABLE test_remove_policy ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' (stage: Complete) Stdout:2025.04.02 03:32:27.431334 [ 6859 ] {9d88c6e7-712e-453d-9129-09ee87d76050} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:27.431820 [ 6859 ] {9d88c6e7-712e-453d-9129-09ee87d76050} default.test_remove_policy (e10d9c41-f3e2-4bde-b1cb-369e85ba48f8): Loading data parts Stdout:2025.04.02 03:32:27.432113 [ 6859 ] {9d88c6e7-712e-453d-9129-09ee87d76050} default.test_remove_policy (e10d9c41-f3e2-4bde-b1cb-369e85ba48f8): There are no data parts Stdout:2025.04.02 03:32:27.432178 [ 6859 ] {9d88c6e7-712e-453d-9129-09ee87d76050} AsyncLoader: Prioritize load job 'startup Atomic database default': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:27.432185 [ 6859 ] {9d88c6e7-712e-453d-9129-09ee87d76050} AsyncLoader: Prioritize load job 'startup Ordinary database default': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:27.439387 [ 6859 ] {9d88c6e7-712e-453d-9129-09ee87d76050} DatabaseAtomic (default): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:27.439421 [ 6859 ] {9d88c6e7-712e-453d-9129-09ee87d76050} DatabaseAtomic (default): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:27.439684 [ 6870 ] {} default.test_remove_policy (e10d9c41-f3e2-4bde-b1cb-369e85ba48f8): Didn't start merge: There are no parts that can be merged. (Collector returned empty ranges set) Stdout:2025.04.02 03:32:27.439884 [ 6859 ] {9d88c6e7-712e-453d-9129-09ee87d76050} TCPHandler: Processed in 0.009186775 sec. Stdout:2025.04.02 03:32:27.440100 [ 6859 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:27.440117 [ 6859 ] {} TCP-Session-8d0b82d0-dfc8-46b9-a41c-04f38826417a: 8d0b82d0-dfc8-46b9-a41c-04f38826417a Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.470594 [ 6859 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63414 Stdout:2025.04.02 03:32:27.470640 [ 6859 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:27.470655 [ 6859 ] {} TCP-Session-8c9b7957-273f-41b3-a0d0-88f2f5fdf905: Authenticating user 'default' from 172.16.8.1:63414 Stdout:2025.04.02 03:32:27.470678 [ 6859 ] {} TCP-Session-8c9b7957-273f-41b3-a0d0-88f2f5fdf905: 8c9b7957-273f-41b3-a0d0-88f2f5fdf905 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.470690 [ 6859 ] {} TCP-Session-8c9b7957-273f-41b3-a0d0-88f2f5fdf905: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.479888 [ 6859 ] {} TCP-Session-8c9b7957-273f-41b3-a0d0-88f2f5fdf905: 8c9b7957-273f-41b3-a0d0-88f2f5fdf905 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:27.480110 [ 6859 ] {3f86ca76-dc05-4e9d-8e88-b0d1a7762344} executeQuery: (from 172.16.8.1:63414) (query 1, line 1) SELECT policy_name FROM system.storage_policies (stage: Complete) Stdout:2025.04.02 03:32:27.480280 [ 6859 ] {3f86ca76-dc05-4e9d-8e88-b0d1a7762344} Planner: Query to stage Complete Stdout:2025.04.02 03:32:27.480429 [ 6859 ] {3f86ca76-dc05-4e9d-8e88-b0d1a7762344} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:27.481155 [ 6859 ] {3f86ca76-dc05-4e9d-8e88-b0d1a7762344} executeQuery: Read 16 rows, 1.58 KiB in 0.001101 sec., 14532.243415077202 rows/sec., 1.40 MiB/sec. Stdout:2025.04.02 03:32:27.481234 [ 6859 ] {3f86ca76-dc05-4e9d-8e88-b0d1a7762344} TCPHandler: Processed in 0.001438598 sec. Stdout:2025.04.02 03:32:27.492273 [ 6859 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:27.492311 [ 6859 ] {} TCP-Session-8c9b7957-273f-41b3-a0d0-88f2f5fdf905: 8c9b7957-273f-41b3-a0d0-88f2f5fdf905 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.538427 [ 6859 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63420 Stdout:2025.04.02 03:32:27.538477 [ 6859 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:27.538491 [ 6859 ] {} TCP-Session-c75d9dd3-1c69-478f-8204-fec7fd0b3e82: Authenticating user 'default' from 172.16.8.1:63420 Stdout:2025.04.02 03:32:27.538514 [ 6859 ] {} TCP-Session-c75d9dd3-1c69-478f-8204-fec7fd0b3e82: c75d9dd3-1c69-478f-8204-fec7fd0b3e82 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.538520 [ 6859 ] {} TCP-Session-c75d9dd3-1c69-478f-8204-fec7fd0b3e82: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.548017 [ 6859 ] {} TCP-Session-c75d9dd3-1c69-478f-8204-fec7fd0b3e82: c75d9dd3-1c69-478f-8204-fec7fd0b3e82 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:27.548243 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} executeQuery: (from 172.16.8.1:63420) (query 1, line 1) SYSTEM RELOAD CONFIG (stage: Complete) Stdout:2025.04.02 03:32:27.550295 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigReloader: Loading config '/etc/clickhouse-server/config.xml' Stdout:2025.04.02 03:32:27.550312 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/config.xml'. Stdout:2025.04.02 03:32:27.551243 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:27.551328 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:27.551408 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_enable_keeper_async_replication.xml'. Stdout:2025.04.02 03:32:27.551490 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_instance_config.xml'. Stdout:2025.04.02 03:32:27.551651 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/logs_config.xml'. Stdout:2025.04.02 03:32:27.551743 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/storage_configuration.xml'. Stdout:2025.04.02 03:32:27.555933 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'. Stdout:2025.04.02 03:32:27.556027 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performing update on configuration Stdout:2025.04.02 03:32:27.562459 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} Application: Changed setting 'max_server_memory_usage' to 27.54 GiB (30.60 GiB available memory * 0.90 max_server_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:27.562470 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.30 GiB (30.60 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:27.562474 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} Application: Merges and mutations memory limit is set to 15.30 GiB Stdout:2025.04.02 03:32:27.563369 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} Application: Setting max_remote_read_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:27.563382 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} Application: Setting max_remote_write_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:27.563387 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} Application: ConcurrencyControl limit is set to 32 CPU slots with 'fair_round_robin' scheduler Stdout:2025.04.02 03:32:27.563398 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} MergeTreeBackgroundExecutor: Loaded new threads count (16) and max tasks count (32) for MergeMutateExecutor Stdout:2025.04.02 03:32:27.563405 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for MoveExecutor Stdout:2025.04.02 03:32:27.563409 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} MergeTreeBackgroundExecutor: Loaded new threads count (16) and max tasks count (16) for FetchExecutor Stdout:2025.04.02 03:32:27.563412 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for CommonExecutor Stdout:2025.04.02 03:32:27.563926 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicy (small_jbod_with_external): Storage policy small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:27.563934 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicySelector: Storage policy `small_jbod_with_external` loaded Stdout:2025.04.02 03:32:27.564054 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicy (one_more_small_jbod_with_external): Storage policy one_more_small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:27.564064 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicySelector: Storage policy `one_more_small_jbod_with_external` loaded Stdout:2025.04.02 03:32:27.564192 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicy (jbods_with_external): Storage policy jbods_with_external created, total volumes 2 Stdout:2025.04.02 03:32:27.564196 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicySelector: Storage policy `jbods_with_external` loaded Stdout:2025.04.02 03:32:27.564307 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicy (moving_jbod_with_external): Storage policy moving_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:27.564311 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicySelector: Storage policy `moving_jbod_with_external` loaded Stdout:2025.04.02 03:32:27.564368 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StorageConfiguration: Volume `small` max_data_part_size is too low (2.00 MiB < 8.00 MiB) Stdout:2025.04.02 03:32:27.564448 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicy (default_disk_with_external): Storage policy default_disk_with_external created, total volumes 2 Stdout:2025.04.02 03:32:27.564452 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicySelector: Storage policy `default_disk_with_external` loaded Stdout:2025.04.02 03:32:27.564607 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StorageConfiguration: Volume `special_warning_small_volume` max_data_part_size is too low (1.00 KiB < 8.00 MiB) Stdout:2025.04.02 03:32:27.564705 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicy (special_warning_policy): Storage policy special_warning_policy created, total volumes 4 Stdout:2025.04.02 03:32:27.564711 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicySelector: Storage policy `special_warning_policy` loaded Stdout:2025.04.02 03:32:27.564735 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} StoragePolicy (default): Storage policy default created, total volumes 1 Stdout:2025.04.02 03:32:27.564813 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} Context: An error has occurred while reloading storage policies, storage policies were not applied: Storage policy `remove_policy_cool_policy` is missing in new configuration Stdout:2025.04.02 03:32:27.564936 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration Stdout:2025.04.02 03:32:27.566759 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigReloader: Loading config '/etc/clickhouse-server/users.xml' Stdout:2025.04.02 03:32:27.567475 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'. Stdout:2025.04.02 03:32:27.567493 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performing update on configuration Stdout:2025.04.02 03:32:27.567982 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performed update on configuration Stdout:2025.04.02 03:32:27.568337 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} MemoryTracker: Query peak memory usage: 10.28 MiB. Stdout:2025.04.02 03:32:27.568404 [ 6859 ] {dd923450-0488-4f40-9cad-f4961aeccba9} TCPHandler: Processed in 0.02048954 sec. Stdout:2025.04.02 03:32:27.568752 [ 6859 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:27.568772 [ 6859 ] {} TCP-Session-c75d9dd3-1c69-478f-8204-fec7fd0b3e82: c75d9dd3-1c69-478f-8204-fec7fd0b3e82 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.605898 [ 6859 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63426 Stdout:2025.04.02 03:32:27.605986 [ 6859 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:27.606013 [ 6859 ] {} TCP-Session-5773fcbf-b34b-4243-b74e-3adb3f69d533: Authenticating user 'default' from 172.16.8.1:63426 Stdout:2025.04.02 03:32:27.606091 [ 6859 ] {} TCP-Session-5773fcbf-b34b-4243-b74e-3adb3f69d533: 5773fcbf-b34b-4243-b74e-3adb3f69d533 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.606101 [ 6859 ] {} TCP-Session-5773fcbf-b34b-4243-b74e-3adb3f69d533: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:27.615192 [ 6859 ] {} TCP-Session-5773fcbf-b34b-4243-b74e-3adb3f69d533: 5773fcbf-b34b-4243-b74e-3adb3f69d533 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:27.615400 [ 6859 ] {f8319be1-9d8b-4f4f-92df-8b538ba7700b} executeQuery: (from 172.16.8.1:63426) (query 1, line 1) SELECT policy_name FROM system.storage_policies (stage: Complete) Stdout:2025.04.02 03:32:27.615610 [ 6859 ] {f8319be1-9d8b-4f4f-92df-8b538ba7700b} Planner: Query to stage Complete Stdout:2025.04.02 03:32:27.615733 [ 6859 ] {f8319be1-9d8b-4f4f-92df-8b538ba7700b} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:27.616421 [ 6859 ] {f8319be1-9d8b-4f4f-92df-8b538ba7700b} executeQuery: Read 16 rows, 1.58 KiB in 0.00106 sec., 15094.33962264151 rows/sec., 1.46 MiB/sec. Stdout:2025.04.02 03:32:27.616505 [ 6859 ] {f8319be1-9d8b-4f4f-92df-8b538ba7700b} TCPHandler: Processed in 0.001547692 sec. Stdout:2025.04.02 03:32:27.626862 [ 6859 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:27.626918 [ 6859 ] {} TCP-Session-5773fcbf-b34b-4243-b74e-3adb3f69d533: 5773fcbf-b34b-4243-b74e-3adb3f69d533 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Executing query DROP TABLE IF EXISTS test_remove_policy on node1 Executing query DROP TABLE IF EXISTS s3_test SYNC on node3 Executing query SYSTEM STOP FETCHES should_execute_table on node2 [gw2] PASSED test_reloading_storage_configuration/test.py::test_remove_policy test_reloading_storage_configuration/test.py::test_remove_volume_from_policy run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] Executing query INSERT INTO should_execute_table SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(200) on node1 Stdout: PID TTY TIME CMD Stdout: 6846 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] thread 1: insert for 2000-01-02: [4, 8, 7, 8, 4, 3, 10, 10, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node3 Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Executing query SELECT value FROM system.build_options WHERE name = 'CXX_FLAGS' on node1 Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Closing connection to 172.16.2.4:2281 Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ['zoo1', 'zoo2', 'zoo3'] Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 [gw3] PASSED test_rename_column/test.py::test_rename_with_parallel_insert test_rename_column/test.py::test_rename_with_parallel_merges http://172.16.9.7:9001 "GET /root?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=data%2F HTTP/1.1" 200 0 Executing query INSERT INTO should_execute_table SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(200) on node1 Executing query CREATE TABLE s3_test ON CLUSTER cluster ( dt Date, id Int64, data String, INDEX min_max (id) TYPE minmax GRANULARITY 3 ) ENGINE=ReplicatedMergeTree() PARTITION BY dt ORDER BY (dt, id) SETTINGS storage_policy='s3' on node1 test_replicated_merge_tree_s3_zero_copy/test.py::test_s3_check_restore Stdout:6846 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_merges SYNC on node1 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_merges SYNC on node2 thread 0: insert for 2000-01-01: [10] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node2 Executing query INSERT INTO should_execute_table SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(200) on node1 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_merges SYNC on node3 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_merges SYNC on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query INSERT INTO s3_test VALUES ('2020-01-02',1,'aScfIPsDOJ'),('2020-01-02',2,'RMpigCgRKx') on node1 Executing query CREATE TABLE test_rename_with_parallel_merges ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_merges', 'node1') ORDER BY num PARTITION BY num % 100 on node1 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk \'{print $1 " " $2}\' /proc/net/route | grep 00000000 | awk \'{print $1}\''] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk '{print $1 " " $2}' /proc/net/route | grep 00000000 | awk '{print $1}'] thread 1: insert for 2000-01-02: [2, 1, 4, 8, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Stdout:eth0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query DETACH TABLE s3_test; on node1 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query CREATE TABLE test_rename_with_parallel_merges ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_merges', 'node2') ORDER BY num PARTITION BY num % 100 on node2 Stdout: eth0: 90978212 8192 0 0 0 0 0 0 234841575 17202 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c grep "^ *eth0:" /proc/net/dev] Stdout: eth0: 90978212 8192 0 0 0 0 0 0 234841575 17202 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query SYSTEM DROP REPLICA '1' FROM TABLE s3_test; on node2 thread 0: insert for 2000-01-01: [4, 2, 1, 1, 10, 8, 1, 9, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Stdout:90978212 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query CREATE TABLE test_rename_with_parallel_merges ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_merges', 'node3') ORDER BY num PARTITION BY num % 100 on node3 Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query INSERT INTO s3_test VALUES ('2020-01-02',1,'cIoudqlfVU'),('2020-01-02',2,'maJqYmSqNH') on node2 Stdout:234841575 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk \'{print $1 " " $2}\' /proc/net/route | grep 00000000 | awk \'{print $1}\''] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk '{print $1 " " $2}' /proc/net/route | grep 00000000 | awk '{print $1}'] Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Stdout:eth0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query ATTACH TABLE s3_test; on node1 Stdout: eth0: 171411937 8488 0 0 0 0 0 0 212379 2306 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep "^ *eth0:" /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c grep "^ *eth0:" /proc/net/dev] Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'grep -a "Code: 210." /var/log/clickhouse-server/clickhouse-server.log | wc -l'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c grep -a "Code: 210." /var/log/clickhouse-server/clickhouse-server.log | wc -l] Executing query CREATE TABLE test_rename_with_parallel_merges ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_merges', 'node4') ORDER BY num PARTITION BY num % 100 on node4 thread 1: insert for 2000-01-02: [7, 9, 6, 5, 10, 1, 10, 3, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SYSTEM RESTORE REPLICA s3_test; on node1 Stdout: eth0: 171411937 8488 0 0 0 0 0 0 212379 2306 0 0 0 0 0 0 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:0 [gw4] PASSED test_reload_client_certificate/test.py::test_correct_cn_cert run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF\n\n \n \n /etc/clickhouse-server/config.d/first_client.crt\n /etc/clickhouse-server/config.d/first_client.key\n true\n true\n sslv2,sslv3\n true\n none\n \n RejectCertificateHandler\n \n \n \n\nEOF'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF /etc/clickhouse-server/config.d/first_client.crt /etc/clickhouse-server/config.d/first_client.key true true sslv2,sslv3 true none RejectCertificateHandler EOF] test_reload_client_certificate/test.py::test_wrong_cn_cert Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance Stdout:171411937 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SET max_partitions_per_insert_block = 10000000; SET max_insert_block_size = 256; INSERT INTO test_rename_with_parallel_merges (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(100) on node1 run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'touch /etc/clickhouse-server/config.d/ssl_conf.xml'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c touch /etc/clickhouse-server/config.d/ssl_conf.xml] Stdout:212379 Executing query SYSTEM START FETCHES should_execute_table on node2 Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Executing query SELECT count() FROM system.replication_queue WHERE table='s3_test' and type='ATTACH_PART' on node1 run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF\n\n \n \n /etc/clickhouse-server/config.d/first_client.crt\n /etc/clickhouse-server/config.d/first_client.key\n true\n true\n sslv2,sslv3\n true\n none\n \n RejectCertificateHandler\n \n \n \n\nEOF'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF /etc/clickhouse-server/config.d/first_client.crt /etc/clickhouse-server/config.d/first_client.key true true sslv2,sslv3 true none RejectCertificateHandler EOF] Executing query INSERT INTO should_execute_table SELECT 0, (select randomPrintableASCII(104857)) FROM numbers(3) on node1 run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'touch /etc/clickhouse-server/config.d/ssl_conf.xml'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c touch /etc/clickhouse-server/config.d/ssl_conf.xml] Executing query DROP TABLE IF EXISTS s3_test SYNC on node1 [gw8] PASSED test_replicated_merge_tree_s3_zero_copy/test.py::test_s3_check_restore Stopping zookeeper node: zoo1 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo1] Executing query INSERT INTO should_execute_table SELECT 1, (select randomPrintableASCII(104857)) FROM numbers(3) on node1 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node1 thread 0: insert for 2000-01-01: [1, 10, 5, 9, 8, 8, 10, 6, 6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO should_execute_table SELECT 2, (select randomPrintableASCII(104857)) FROM numbers(3) on node1 Executing query DROP TABLE IF EXISTS s3_test SYNC on node2 Executing query INSERT INTO should_execute_table SELECT 3, (select randomPrintableASCII(104857)) FROM numbers(3) on node1 Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node2 Executing query INSERT INTO should_execute_table SELECT 4, (select randomPrintableASCII(104857)) FROM numbers(3) on node1 Executing query DROP TABLE IF EXISTS s3_test SYNC on node3 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query INSERT INTO should_execute_table SELECT 5, (select randomPrintableASCII(104857)) FROM numbers(3) on node1 No clickhouse process running. Start new one. Executing query DROP TABLE IF EXISTS test_drop_table SYNC on node3 http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/cc46f160a66ecef9ac39da9df8bc99dc50a711a8bdf7797951ac21a8e1c9f8e3/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/cc46f160a66ecef9ac39da9df8bc99dc50a711a8bdf7797951ac21a8e1c9f8e3/json HTTP/1.1" 200 586 Executing query INSERT INTO should_execute_table SELECT 6, (select randomPrintableASCII(104857)) FROM numbers(3) on node1 thread 0: insert for 2000-01-01: [2, 5, 2, 6, 9, 1, 2, 9, 4, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 http://172.16.9.7:9001 "GET /root?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=data%2F HTTP/1.1" 200 0 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env --project-name roottestreplicatedmergetrees3zerocopy-gw8 --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/docker-compose.yml stop --timeout 20] Executing query INSERT INTO should_execute_table SELECT 7, (select randomPrintableASCII(104857)) FROM numbers(3) on node1 Executing query INSERT INTO should_execute_table SELECT 8, (select randomPrintableASCII(104857)) FROM numbers(3) on node1 thread 1: insert for 2000-01-02: [5, 7, 3, 4, 6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Stopped Stopping zookeeper node: zoo2 Command:[docker compose --project-name roottestremovestalemovingparts-gw1 --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop zoo2] Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopped Stopping zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo2] Executing query INSERT INTO should_execute_table SELECT 9, (select randomPrintableASCII(104857)) FROM numbers(3) on node1 thread 0: insert for 2000-01-01: [9, 3, 1, 5, 3, 9, 4, 5, 9, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:91031758 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:258838040 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:195355350 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:244217 Executing query SELECT count() FROM system.replicated_fetches on node2 Executing query SET max_partitions_per_insert_block = 10000000; SET max_insert_block_size = 256; INSERT INTO test_rename_with_parallel_merges (num,num2) SELECT number + 100 AS num, number + 1 + 100 AS num2 FROM numbers_mt(100) on node1 Executing query SELECT count() FROM system.replication_queue WHERE postpone_reason like '%fetches have already throttled%' on node2 thread 0: insert for 2000-01-01: [4, 8, 10, 10, 4, 8, 1, 7, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [1, 4, 10, 5, 5, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 1: insert for 2000-01-02: [5, 5, 8, 4, 9, 6, 10, 2, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:7614 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:7614 Executing query select 20 on node1 Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopped Stopping zookeeper node: zoo3 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo3] thread 0: insert for 2000-01-01: [3, 2, 3, 10, 10, 6, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:91032682 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:261068484 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:197774549 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:252258 Executing query SELECT count() FROM system.replicated_fetches on node2 thread 1: insert for 2000-01-02: [10, 5, 9, 4, 8, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT count() FROM system.replication_queue WHERE postpone_reason like '%fetches have already throttled%' on node2 Executing query SET max_partitions_per_insert_block = 10000000; SET max_insert_block_size = 256; INSERT INTO test_rename_with_parallel_merges (num,num2) SELECT number + 200 AS num, number + 1 + 200 AS num2 FROM numbers_mt(100) on node1 thread 0: insert for 2000-01-01: [4, 3, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [7, 8, 7, 7, 10, 6, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopped Starting zookeeper node: zoo1 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo1] thread 0: insert for 2000-01-01: [8, 4, 9, 8, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [10, 1, 1, 4, 9, 6, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:91034200 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SET max_partitions_per_insert_block = 10000000; SET max_insert_block_size = 256; INSERT INTO test_rename_with_parallel_merges (num,num2) SELECT number + 300 AS num, number + 1 + 300 AS num2 FROM numbers_mt(100) on node1 Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Started Starting zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo2] Stdout:265266610 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:203104262 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:260902 Executing query SELECT count() FROM system.replicated_fetches on node2 Executing query SELECT count() FROM system.replication_queue WHERE postpone_reason like '%fetches have already throttled%' on node2 thread 0: insert for 2000-01-01: [6, 5, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [10, 8, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Started Starting zookeeper node: zoo3 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo3] Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 0: insert for 2000-01-01: [3, 9, 9, 9, 3, 2, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Started get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.2.2, port:2281, use_ssl:True Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused thread 1: insert for 2000-01-02: [10, 2, 4, 3, 7, 7, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SET max_partitions_per_insert_block = 10000000; SET max_insert_block_size = 256; INSERT INTO test_rename_with_parallel_merges (num,num2) SELECT number + 400 AS num, number + 1 + 400 AS num2 FROM numbers_mt(100) on node1 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Stdout:91035586 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance thread 0: insert for 2000-01-01: [8, 5, 1, 6, 6, 10, 4, 9, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:270132064 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Stdout:207134303 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance Executing query CREATE TABLE test_remove_volume_from_policy ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 Stdout:269120 Executing query SELECT count() FROM system.replicated_fetches on node2 Executing query SELECT volume_name FROM system.storage_policies WHERE policy_name = 'test_remove_volume_from_policy_cool_policy' on node1 Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance Executing query SELECT count() FROM system.replication_queue WHERE postpone_reason like '%fetches have already throttled%' on node2 Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Executing query SELECT disks FROM system.storage_policies WHERE policy_name = 'test_remove_volume_from_policy_cool_policy' on node1 thread 1: insert for 2000-01-02: [2, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SYSTEM RELOAD CONFIG on node1 thread 0: insert for 2000-01-01: [5, 10, 5, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT volume_name FROM system.storage_policies WHERE policy_name = 'test_remove_volume_from_policy_cool_policy' on node1 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query SELECT disks FROM system.storage_policies WHERE policy_name = 'test_remove_volume_from_policy_cool_policy' on node1 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c cat /var/log/clickhouse-server/clickhouse-server.log] Stdout:2025.04.02 03:32:29.105675 [ 7614 ] {} SentryWriter: Sending crash reports is disabled Stdout:2025.04.02 03:32:29.162970 [ 7614 ] {} Application: Starting ClickHouse 25.2.2.96963.altinityantalya.96963 (revision: 54496, git hash: 25f7c2a766916d92d37aee82d9defa5f14b86ec8, build id: 4783FC52DD13D5DFB0294BDDD711047195FEB5A6), PID 7614 Stdout:2025.04.02 03:32:29.163122 [ 7614 ] {} Application: starting up Stdout:2025.04.02 03:32:29.163134 [ 7614 ] {} Application: OS name: Linux, version: 5.15.0-130-generic, architecture: x86_64 Stdout:2025.04.02 03:32:29.163291 [ 7614 ] {} Jemalloc: Value for background_thread set to true (from true) Stdout:2025.04.02 03:32:29.168532 [ 7614 ] {} Application: Available RAM: 30.60 GiB; logical cores: 16; used cores: 16. Stdout:2025.04.02 03:32:29.168560 [ 7614 ] {} Application: Available CPU instruction sets: SSE, SSE2, SSE3, SSSE3, SSE41, SSE42, F16C, POPCNT, BMI1, BMI2, PCLMUL, AES, AVX, FMA, AVX2, SHA, ADX, RDRAND, RDSEED, RDTSCP, CLFLUSHOPT, CLWB, XSAVE, OSXSAVE Stdout:2025.04.02 03:32:29.168600 [ 7614 ] {} Pipe: Pipe capacity is 1.00 MiB Stdout:2025.04.02 03:32:29.169430 [ 7614 ] {} CgroupsReader: Will create cgroup reader from '/sys/fs/cgroup/' (cgroups version: v2) Stdout:2025.04.02 03:32:29.169605 [ 7614 ] {} AsynchronousMetrics: Scanning /sys/class/thermal Stdout:2025.04.02 03:32:29.169619 [ 7614 ] {} AsynchronousMetrics: Scanning /sys/block Stdout:2025.04.02 03:32:29.169701 [ 7614 ] {} AsynchronousMetrics: Scanning /sys/devices/system/edac Stdout:2025.04.02 03:32:29.169720 [ 7614 ] {} AsynchronousMetrics: Scanning /sys/class/hwmon Stdout:2025.04.02 03:32:29.169799 [ 7614 ] {} StatusFile: Status file /var/lib/clickhouse/status already exists - unclean restart. Contents: Stdout:PID: 6846 Stdout:Started at: 2025-04-02 03:32:24 Stdout:Revision: 54496 Stdout: Stdout:2025.04.02 03:32:29.333246 [ 7614 ] {} Application: Integrity check of the executable successfully passed (checksum: 2D3F861318CFEDFBBB742B17F83CC397) Stdout:2025.04.02 03:32:29.333339 [ 7614 ] {} Application: Will do mlock to prevent executable memory from being paged out. It may take a few seconds. Stdout:2025.04.02 03:32:29.337278 [ 7614 ] {} Application: The memory map of clickhouse executable has been mlock'ed, total 287.17 MiB Stdout:2025.04.02 03:32:29.337314 [ 7614 ] {} MemoryWorker: Starting background memory thread with period of 50ms, using Cgroups as source Stdout:2025.04.02 03:32:29.337384 [ 7614 ] {} BackgroundSchedulePool/BgSchPool: Create BackgroundSchedulePool with 512 threads Stdout:2025.04.02 03:32:29.393470 [ 7629 ] {} MemoryTracker: Correcting the value of global memory tracker from 3.22 MiB to 105.89 MiB Stdout:2025.04.02 03:32:29.400333 [ 7614 ] {} Application: rlimit on number of file descriptors is 1048576 Stdout:2025.04.02 03:32:29.400357 [ 7614 ] {} Application: rlimit on number of threads is 18446744073709551615 Stdout:2025.04.02 03:32:29.400329 [ 7631 ] {} CancellationChecker: Started worker function Stdout:2025.04.02 03:32:29.400373 [ 7614 ] {} Application: Initializing DateLUT. Stdout:2025.04.02 03:32:29.400395 [ 7614 ] {} Application: Initialized DateLUT with time zone 'Etc/UTC'. Stdout:2025.04.02 03:32:29.400428 [ 7614 ] {} Context: Setting up /var/lib/clickhouse/tmp/ to store temporary data in it Stdout:2025.04.02 03:32:29.400654 [ 7614 ] {} Application: Configuration parameter 'interserver_http_host' doesn't exist or exists and empty. Will use 'node1' as replica host. Stdout:2025.04.02 03:32:29.400672 [ 7614 ] {} Application: Initializing interserver credentials. Stdout:2025.04.02 03:32:29.400781 [ 7614 ] {} NamedCollectionsMetadataStorage: Using local storage for named collections at path: /var/lib/clickhouse/named_collections Stdout:2025.04.02 03:32:29.400838 [ 7614 ] {} FileCacheFactory: Will load 0 caches from default cache config Stdout:2025.04.02 03:32:29.402688 [ 7614 ] {} ConfigReloader: Loading config '/etc/clickhouse-server/config.xml' Stdout:2025.04.02 03:32:29.402698 [ 7614 ] {} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/config.xml'. Stdout:2025.04.02 03:32:29.403592 [ 7614 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:29.403696 [ 7614 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:29.403800 [ 7614 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_enable_keeper_async_replication.xml'. Stdout:2025.04.02 03:32:29.403869 [ 7614 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_instance_config.xml'. Stdout:2025.04.02 03:32:29.404026 [ 7614 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/logs_config.xml'. Stdout:2025.04.02 03:32:29.404122 [ 7614 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/storage_configuration.xml'. Stdout:2025.04.02 03:32:29.408540 [ 7614 ] {} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'. Stdout:2025.04.02 03:32:29.408652 [ 7614 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performing update on configuration Stdout:2025.04.02 03:32:29.415786 [ 7614 ] {} Application: Changed setting 'max_server_memory_usage' to 27.54 GiB (30.60 GiB available memory * 0.90 max_server_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:29.415807 [ 7614 ] {} Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.30 GiB (30.60 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:29.415811 [ 7614 ] {} Application: Merges and mutations memory limit is set to 15.30 GiB Stdout:2025.04.02 03:32:29.415932 [ 7614 ] {} Application: Setting max_remote_read_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:29.415943 [ 7614 ] {} Application: Setting max_remote_write_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:29.415950 [ 7614 ] {} Application: ConcurrencyControl limit is set to 32 CPU slots with 'fair_round_robin' scheduler Stdout:2025.04.02 03:32:29.415965 [ 7614 ] {} BackgroundSchedulePool/BgBufSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:29.417442 [ 7614 ] {} BackgroundSchedulePool/BgMBSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:29.420289 [ 7614 ] {} BackgroundSchedulePool/BgDistSchPool: Create BackgroundSchedulePool with 16 threads Stdout:2025.04.02 03:32:29.421946 [ 7614 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration Stdout:2025.04.02 03:32:29.421972 [ 7614 ] {} ConfigReloader: Config reload interval set to 2000ms Stdout:2025.04.02 03:32:29.422346 [ 7614 ] {} Application: Listening for replica communication (interserver): http://0.0.0.0:9009 Stdout:2025.04.02 03:32:29.424608 [ 7614 ] {} ConfigReloader: Loading config '/etc/clickhouse-server/users.xml' Stdout:2025.04.02 03:32:29.424620 [ 7614 ] {} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/users.xml'. Stdout:2025.04.02 03:32:29.425254 [ 7614 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:29.425300 [ 7614 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:29.425350 [ 7614 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/users.d/0_common_instance_users.xml'. Stdout:2025.04.02 03:32:29.425380 [ 7614 ] {} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/users.d/0_random_settings.xml'. Stdout:2025.04.02 03:32:29.425854 [ 7614 ] {} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'. Stdout:2025.04.02 03:32:29.425873 [ 7614 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performing update on configuration Stdout:2025.04.02 03:32:29.426385 [ 7614 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performed update on configuration Stdout:2025.04.02 03:32:29.426397 [ 7614 ] {} ConfigReloader: Config reload interval set to 2000ms Stdout:2025.04.02 03:32:29.426662 [ 7614 ] {} Access(user directories): Added users_xml access storage 'users_xml', path: /etc/clickhouse-server/users.xml Stdout:2025.04.02 03:32:29.426770 [ 7614 ] {} Access(user directories): Added local_directory access storage 'local_directory', path: /var/lib/clickhouse/access/ Stdout:2025.04.02 03:32:29.427264 [ 7614 ] {} CgroupsMemoryUsageObserver: Started cgroup current memory usage observer thread Stdout:2025.04.02 03:32:29.427448 [ 8197 ] {} CgroupsMemoryUsageObserver: Memory amount initially available to the process is 30.60 GiB Stdout:2025.04.02 03:32:29.429417 [ 7614 ] {} Context: Initialized background executor for merges and mutations with num_threads=16, num_tasks=32, scheduling_policy=round_robin Stdout:2025.04.02 03:32:29.430204 [ 7614 ] {} Context: Initialized background executor for move operations with num_threads=8, num_tasks=8 Stdout:2025.04.02 03:32:29.431594 [ 7614 ] {} Context: Initialized background executor for fetches with num_threads=16, num_tasks=16 Stdout:2025.04.02 03:32:29.432289 [ 7614 ] {} Context: Initialized background executor for common operations (e.g. clearing old parts) with num_threads=8, num_tasks=8 Stdout:2025.04.02 03:32:29.435882 [ 7614 ] {} Context: Delay accounting is not enabled, OSIOWaitMicroseconds will not be gathered. You can enable it using `echo 1 > /proc/sys/kernel/task_delayacct` or by using sysctl. Stdout:2025.04.02 03:32:29.436212 [ 7614 ] {} DNSCacheUpdater: Update period 15 seconds Stdout:2025.04.02 03:32:29.436250 [ 7634 ] {} DNSResolver: Updating DNS cache Stdout:2025.04.02 03:32:29.436276 [ 7634 ] {} DNSResolver: Updated DNS cache Stdout:2025.04.02 03:32:29.436305 [ 7614 ] {} Application: Loading metadata from /var/lib/clickhouse/ Stdout:2025.04.02 03:32:29.437941 [ 7614 ] {} Context: Database disk name: default Stdout:2025.04.02 03:32:29.437957 [ 7614 ] {} Context: Database disk name: default, path: /var/lib/clickhouse/ Stdout:2025.04.02 03:32:29.437990 [ 7614 ] {} loadSystemDatabase: metadata_file_path metadata/system.sql, existsFile true Stdout:2025.04.02 03:32:29.438138 [ 7614 ] {} loadSystemDatabase: metadata_file_path metadata/information_schema.sql, existsFile true Stdout:2025.04.02 03:32:29.438190 [ 7614 ] {} loadSystemDatabase: metadata_file_path metadata/INFORMATION_SCHEMA.sql, existsFile true Stdout:2025.04.02 03:32:29.438275 [ 7614 ] {} DatabaseAtomic (system): Metadata processed, database system has 0 tables and 0 dictionaries in total. Stdout:2025.04.02 03:32:29.438283 [ 7614 ] {} TablesLoader: Parsed metadata of 0 tables in 1 databases in 6.2487e-05 sec Stdout:2025.04.02 03:32:29.438288 [ 7614 ] {} ReferentialDeps: No tables thread 1: insert for 2000-01-02: [8, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:2025.04.02 03:32:29.438290 [ 7614 ] {} LoadingDeps: No tables Stdout:2025.04.02 03:32:29.438329 [ 7614 ] {} AsyncLoader: Prioritize load job 'startup Atomic database system': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:29.438335 [ 7614 ] {} AsyncLoader: Prioritize load job 'startup Ordinary database system': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:29.438343 [ 7614 ] {} AsyncLoader: Schedule load job 'startup Ordinary database system' into ForegroundLoad Stdout:2025.04.02 03:32:29.438347 [ 7614 ] {} AsyncLoader: Schedule load job 'startup Atomic database system' into ForegroundLoad Stdout:2025.04.02 03:32:29.438351 [ 7614 ] {} AsyncLoader: Change current priority: none -> 0 Stdout:2025.04.02 03:32:29.438355 [ 7614 ] {} AsyncLoader: Spawn loader worker #1 in ForegroundLoad Stdout:2025.04.02 03:32:29.438410 [ 7614 ] {} AsyncLoader: Wait load job 'startup Atomic database system' in ForegroundLoad Stdout:2025.04.02 03:32:29.438492 [ 8264 ] {} AsyncLoader: Execute load job 'startup Ordinary database system' in ForegroundLoad Stdout:2025.04.02 03:32:29.438533 [ 8264 ] {} AsyncLoader: Finish load job 'startup Ordinary database system' with status OK Stdout:2025.04.02 03:32:29.438547 [ 8264 ] {} AsyncLoader: Spawn loader worker #2 in ForegroundLoad Stdout:2025.04.02 03:32:29.438624 [ 8264 ] {} AsyncLoader: Execute load job 'startup Atomic database system' in ForegroundLoad Stdout:2025.04.02 03:32:29.438650 [ 8264 ] {} AsyncLoader: Finish load job 'startup Atomic database system' with status OK Stdout:2025.04.02 03:32:29.438661 [ 8264 ] {} AsyncLoader: Stop worker in ForegroundLoad Stdout:2025.04.02 03:32:29.438695 [ 7614 ] {} SystemLog: Creating system.query_log from query_log Stdout:2025.04.02 03:32:29.438712 [ 8265 ] {} AsyncLoader: Stop worker in ForegroundLoad Stdout:2025.04.02 03:32:29.438754 [ 8265 ] {} AsyncLoader: Change current priority: 0 -> none Stdout:2025.04.02 03:32:29.448565 [ 7614 ] {} SystemLog: Creating system.query_thread_log from query_thread_log Stdout:2025.04.02 03:32:29.449151 [ 7614 ] {} SystemLog: Creating system.part_log from part_log Stdout:2025.04.02 03:32:29.449486 [ 7614 ] {} SystemLog: Creating system.trace_log from trace_log Stdout:2025.04.02 03:32:29.449755 [ 7614 ] {} SystemLog: Creating system.crash_log from crash_log Stdout:2025.04.02 03:32:29.449899 [ 7614 ] {} SystemLog: Creating system.text_log from text_log Stdout:2025.04.02 03:32:29.450185 [ 7614 ] {} SystemLog: Creating system.metric_log from metric_log Stdout:2025.04.02 03:32:29.455213 [ 7614 ] {} SystemLog: Creating system.latency_log from latency_log Stdout:2025.04.02 03:32:29.455479 [ 7614 ] {} SystemLog: Creating system.error_log from error_log Stdout:2025.04.02 03:32:29.455761 [ 7614 ] {} SystemLog: Not creating system.filesystem_cache_log since corresponding section 'filesystem_cache_log' is missing from config Stdout:2025.04.02 03:32:29.455811 [ 7614 ] {} SystemLog: Not creating system.filesystem_read_prefetches_log since corresponding section 'filesystem_read_prefetches_log' is missing from config Stdout:2025.04.02 03:32:29.455831 [ 7614 ] {} SystemLog: Creating system.s3queue_log from s3queue_log Stdout:2025.04.02 03:32:29.456241 [ 7614 ] {} SystemLog: Not creating system.azure_queue_log since corresponding section 'azure_queue_log' is missing from config Stdout:2025.04.02 03:32:29.456259 [ 7614 ] {} SystemLog: Creating system.asynchronous_metric_log from asynchronous_metric_log Stdout:2025.04.02 03:32:29.456448 [ 7614 ] {} SystemLog: Creating system.opentelemetry_span_log from opentelemetry_span_log Stdout:2025.04.02 03:32:29.456729 [ 7614 ] {} SystemLog: Creating system.query_views_log from query_views_log Stdout:2025.04.02 03:32:29.457012 [ 7614 ] {} SystemLog: Not creating system.zookeeper_log since corresponding section 'zookeeper_log' is missing from config Stdout:2025.04.02 03:32:29.457027 [ 7614 ] {} SystemLog: Not creating system.session_log since corresponding section 'session_log' is missing from config Stdout:2025.04.02 03:32:29.457036 [ 7614 ] {} SystemLog: Not creating system.transactions_info_log since corresponding section 'transactions_info_log' is missing from config Stdout:2025.04.02 03:32:29.457060 [ 7614 ] {} SystemLog: Creating system.processors_profile_log from processors_profile_log Stdout:2025.04.02 03:32:29.457356 [ 7614 ] {} SystemLog: Creating system.asynchronous_insert_log from asynchronous_insert_log Stdout:2025.04.02 03:32:29.457599 [ 7614 ] {} SystemLog: Creating system.backup_log from backup_log Stdout:2025.04.02 03:32:29.457856 [ 7614 ] {} SystemLog: Creating system.blob_storage_log from blob_storage_log Stdout:2025.04.02 03:32:29.458086 [ 7614 ] {} SystemLog: Creating system.query_metric_log from query_metric_log Stdout:2025.04.02 03:32:29.463114 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.463126 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.464849 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.464865 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.464886 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.464890 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.464906 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.464910 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.464924 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.464927 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465001 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465007 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465115 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465121 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465161 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465164 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465206 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465209 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465246 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465250 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465279 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465287 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465321 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465325 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465355 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465358 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465390 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465393 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465423 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465426 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465459 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465464 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465485 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465489 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465508 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465511 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465536 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465539 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465562 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465565 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465585 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465590 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465608 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465611 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465628 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465631 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465655 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465659 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465677 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465681 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465728 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465734 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465758 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465761 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465919 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465926 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.465953 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.465956 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466012 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466018 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466040 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466056 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466086 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466089 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466127 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466131 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466165 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466170 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466211 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466220 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466264 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466270 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466333 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466338 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466405 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466411 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466441 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466445 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466530 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466535 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466561 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466565 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466581 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466585 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466606 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466609 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466640 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466644 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466663 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466666 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466684 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466688 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466731 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466735 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466766 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466769 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466793 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466796 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.466933 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.466938 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467016 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467025 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467071 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467075 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467093 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467096 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467132 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467136 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467177 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467181 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467202 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467205 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467221 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467224 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467342 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467347 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467508 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467517 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467578 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467583 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467687 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467692 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467784 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467789 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467833 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467836 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.467868 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.467872 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468005 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468011 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468075 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468080 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468120 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468124 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468159 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468163 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468197 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468201 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468319 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468325 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468381 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468385 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468435 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468439 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468470 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468473 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468523 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468527 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468557 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468563 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468610 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468614 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468682 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468688 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468709 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468712 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468740 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468743 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468774 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468780 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468809 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468812 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468842 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468845 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468912 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468923 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.468980 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.468992 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469022 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469029 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469075 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469080 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469105 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469109 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469188 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469194 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469264 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469269 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469291 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469300 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469328 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469332 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469356 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469360 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469383 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469386 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469406 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469410 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469427 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469433 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469470 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469473 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469496 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469499 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469519 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469522 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469556 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469559 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469595 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469603 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.469677 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.469681 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.477081 [ 7614 ] {} DatabaseCatalog: Found 9 partially dropped tables. Will load them and retry removal. Stdout:2025.04.02 03:32:29.477304 [ 8265 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174) from metadata_dropped/default.test_add_disk.ab1b9e7f-3e87-42e0-987c-542af287b174.sql Stdout:2025.04.02 03:32:29.477389 [ 8286 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3) from metadata_dropped/default.test_add_disk.8a487b06-210e-4c1c-91cf-235fb0fcffb3.sql Stdout:2025.04.02 03:32:29.477712 [ 8287 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f) from metadata_dropped/default.test_add_disk_to_policy.e1af3e60-fe6e-4895-a02b-fced2189743f.sql Stdout:2025.04.02 03:32:29.478025 [ 8265 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:29.478185 [ 8288 ] {} DatabaseCatalog: Trying load partially dropped table default.test_new_policy_works (9288e96b-23dc-4115-80b9-90a084e01018) from metadata_dropped/default.test_new_policy_works.9288e96b-23dc-4115-80b9-90a084e01018.sql Stdout:2025.04.02 03:32:29.478280 [ 8291 ] {} DatabaseCatalog: Trying load partially dropped table default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce) from metadata_dropped/default.test_remove_disk.b18b3533-cce9-430c-ba4b-cbd718231cce.sql Stdout:2025.04.02 03:32:29.480891 [ 8292 ] {} DatabaseCatalog: Trying load partially dropped table default.test_remove_disk_from_policy (ee9c3908-42fe-4bec-94ec-e0eae1e532db) from metadata_dropped/default.test_remove_disk_from_policy.ee9c3908-42fe-4bec-94ec-e0eae1e532db.sql Stdout:2025.04.02 03:32:29.481191 [ 8292 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:29.481691 [ 8292 ] {} StoragePolicy (small_jbod_with_external): Storage policy small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:29.481704 [ 8292 ] {} StoragePolicySelector: Storage policy `small_jbod_with_external` loaded Stdout:2025.04.02 03:32:29.481816 [ 8292 ] {} StoragePolicy (one_more_small_jbod_with_external): Storage policy one_more_small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:29.481820 [ 8292 ] {} StoragePolicySelector: Storage policy `one_more_small_jbod_with_external` loaded Stdout:2025.04.02 03:32:29.481995 [ 8286 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:29.482180 [ 8292 ] {} StoragePolicy (jbods_with_external): Storage policy jbods_with_external created, total volumes 2 Stdout:2025.04.02 03:32:29.482191 [ 8292 ] {} StoragePolicySelector: Storage policy `jbods_with_external` loaded Stdout:2025.04.02 03:32:29.478357 [ 8290 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5) from metadata_dropped/default.test_add_volume_to_policy.b03aef8f-055b-454d-bac3-7055c6fd12b5.sql Stdout:2025.04.02 03:32:29.482274 [ 8287 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:29.482330 [ 8292 ] {} StoragePolicy (moving_jbod_with_external): Storage policy moving_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:29.482341 [ 8292 ] {} StoragePolicySelector: Storage policy `moving_jbod_with_external` loaded Stdout:2025.04.02 03:32:29.482540 [ 8292 ] {} StorageConfiguration: Volume `small` max_data_part_size is too low (2.00 MiB < 8.00 MiB) Stdout:2025.04.02 03:32:29.482514 [ 8293 ] {} DatabaseCatalog: Trying load partially dropped table default.test_remove_policy (e10d9c41-f3e2-4bde-b1cb-369e85ba48f8) from metadata_dropped/default.test_remove_policy.e10d9c41-f3e2-4bde-b1cb-369e85ba48f8.sql Stdout:2025.04.02 03:32:29.482619 [ 8288 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:29.482121 [ 8289 ] {} DatabaseCatalog: Trying load partially dropped table default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c) from metadata_dropped/default.test_add_policy.d69b6b75-0af1-4149-a0f9-c13bb05ea67c.sql Stdout:2025.04.02 03:32:29.482637 [ 8291 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:29.482649 [ 8290 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:29.482771 [ 8292 ] {} StoragePolicy (default_disk_with_external): Storage policy default_disk_with_external created, total volumes 2 Stdout:2025.04.02 03:32:29.482787 [ 8292 ] {} StoragePolicySelector: Storage policy `default_disk_with_external` loaded Stdout:2025.04.02 03:32:29.482798 [ 8293 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:29.483000 [ 8292 ] {} StorageConfiguration: Volume `special_warning_small_volume` max_data_part_size is too low (1.00 KiB < 8.00 MiB) Stdout:2025.04.02 03:32:29.483193 [ 8289 ] {} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:29.484276 [ 8292 ] {} StoragePolicy (special_warning_policy): Storage policy special_warning_policy created, total volumes 4 Stdout:2025.04.02 03:32:29.484294 [ 8292 ] {} StoragePolicySelector: Storage policy `special_warning_policy` loaded Stdout:2025.04.02 03:32:29.484412 [ 8292 ] {} StoragePolicy (test_remove_volume_from_policy_cool_policy): Storage policy test_remove_volume_from_policy_cool_policy created, total volumes 2 Stdout:2025.04.02 03:32:29.484420 [ 8292 ] {} StoragePolicySelector: Storage policy `test_remove_volume_from_policy_cool_policy` loaded Stdout:2025.04.02 03:32:29.484438 [ 8292 ] {} StoragePolicy (default): Storage policy default created, total volumes 1 Stdout:2025.04.02 03:32:29.484549 [ 8292 ] {} default.test_remove_disk_from_policy (ee9c3908-42fe-4bec-94ec-e0eae1e532db): Loading data parts Stdout:2025.04.02 03:32:29.484650 [ 8287 ] {} default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f): Loading data parts Stdout:2025.04.02 03:32:29.484772 [ 8289 ] {} default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c): Loading data parts Stdout:2025.04.02 03:32:29.484958 [ 8290 ] {} default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5): Loading data parts Stdout:2025.04.02 03:32:29.484995 [ 8286 ] {} default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3): Loading data parts Stdout:2025.04.02 03:32:29.485499 [ 8292 ] {} default.test_remove_disk_from_policy (ee9c3908-42fe-4bec-94ec-e0eae1e532db): There are no data parts Stdout:2025.04.02 03:32:29.485558 [ 8293 ] {} default.test_remove_policy (e10d9c41-f3e2-4bde-b1cb-369e85ba48f8): Loading data parts Stdout:2025.04.02 03:32:29.486263 [ 8291 ] {} default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce): Loading data parts Stdout:2025.04.02 03:32:29.486590 [ 8288 ] {} DatabaseCatalog: Cannot load partially dropped table default.test_new_policy_works (9288e96b-23dc-4115-80b9-90a084e01018) from: metadata_dropped/default.test_new_policy_works.9288e96b-23dc-4115-80b9-90a084e01018.sql. Parsed query: ATTACH TABLE default.test_new_policy_works UUID '9288e96b-23dc-4115-80b9-90a084e01018' (`d` UInt64) ENGINE = MergeTree ORDER BY d SETTINGS storage_policy = 'cool_policy', index_granularity = 8192. Will remove metadata and store/928/9288e96b-23dc-4115-80b9-90a084e01018/. Garbage may be left in ZooKeeper.: Code: 478. DB::Exception: Unknown storage policy `cool_policy`. (UNKNOWN_POLICY), Stack trace (when copying this message, always include the lines below): Stdout: Stdout:0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000cfb8fbb Stdout:1. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x0000000007ff830c Stdout:2. DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x0000000007ff802b Stdout:3. DB::StoragePolicySelector::get(String const&) const @ 0x00000000105c18e1 Stdout:4. DB::Context::getStoragePolicy(String const&) const @ 0x0000000010b34cbf Stdout:5. DB::MergeTreeData::getStoragePolicy() const @ 0x000000001201df88 Stdout:6. DB::MergeTreeData::initializeDirectoriesAndFormatVersion(String const&, bool, String const&, bool) @ 0x00000000120136a8 Stdout:7. DB::StorageMergeTree::StorageMergeTree(DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, DB::LoadingStrictnessLevel, std::shared_ptr, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr>) @ 0x00000000123e6548 Stdout:8. DB::create(DB::StorageFactory::Arguments const&) @ 0x00000000123e2c8c Stdout:9. DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr, std::shared_ptr, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, DB::LoadingStrictnessLevel) const @ 0x00000000119e37bf Stdout:10. DB::createTableFromAST(DB::ASTCreateQuery, String const&, String const&, std::shared_ptr, DB::LoadingStrictnessLevel) @ 0x00000000104454aa Stdout:11. DB::DatabaseCatalog::enqueueDroppedTableCleanup(DB::StorageID, std::shared_ptr, String, bool) @ 0x0000000010c038a9 Stdout:12. void std::__function::__policy_invoker::__call_impl[abi:ne190107]>(std::__function::__policy_storage const*) @ 0x0000000010c14b0e Stdout:13. void DB::ThreadPoolCallbackRunnerLocal>, std::function>::executeCallback>(std::promise&, std::function&&) @ 0x000000000efb3e95 Stdout:14. DB::ThreadPoolCallbackRunnerLocal>, std::function>::operator()(std::function&&, Priority, std::optional)::'lambda'()::operator()() @ 0x000000000efb3c9b Stdout:15. ThreadPoolImpl>::ThreadFromThreadPool::worker() @ 0x000000000d0992db Stdout:16. void std::__function::__policy_invoker::__call_impl[abi:ne190107]::ThreadFromGlobalPoolImpl>::ThreadFromThreadPool::*)(), ThreadPoolImpl>::ThreadFromThreadPool*>(void (ThreadPoolImpl>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl>::ThreadFromThreadPool*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000d09fda3 Stdout:17. ThreadPoolImpl::ThreadFromThreadPool::worker() @ 0x000000000d09656f Stdout:18. void* std::__thread_proxy[abi:ne190107]>, void (ThreadPoolImpl::ThreadFromThreadPool::*)(), ThreadPoolImpl::ThreadFromThreadPool*>>(void*) @ 0x000000000d09d85a Stdout:19. ? @ 0x00007f693114aac3 Stdout:20. ? @ 0x00007f69311dc850 Stdout: (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:29.486891 [ 8265 ] {} default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174): Loading data parts Stdout:2025.04.02 03:32:29.486973 [ 8293 ] {} default.test_remove_policy (e10d9c41-f3e2-4bde-b1cb-369e85ba48f8): There are no data parts Stdout:2025.04.02 03:32:29.487189 [ 8289 ] {} default.test_add_policy (d69b6b75-0af1-4149-a0f9-c13bb05ea67c): There are no data parts Stdout:2025.04.02 03:32:29.488818 [ 8287 ] {} default.test_add_disk_to_policy (e1af3e60-fe6e-4895-a02b-fced2189743f): There are no data parts Stdout:2025.04.02 03:32:29.489206 [ 8286 ] {} default.test_add_disk (8a487b06-210e-4c1c-91cf-235fb0fcffb3): There are no data parts Stdout:2025.04.02 03:32:29.489211 [ 8290 ] {} default.test_add_volume_to_policy (b03aef8f-055b-454d-bac3-7055c6fd12b5): There are no data parts Stdout:2025.04.02 03:32:29.489412 [ 8265 ] {} default.test_add_disk (ab1b9e7f-3e87-42e0-987c-542af287b174): There are no data parts Stdout:2025.04.02 03:32:29.490835 [ 8291 ] {} default.test_remove_disk (b18b3533-cce9-430c-ba4b-cbd718231cce): There are no data parts Stdout:2025.04.02 03:32:29.491124 [ 7614 ] {} DatabaseAtomic (default): Metadata processed, database default has 0 tables and 0 dictionaries in total. Stdout:2025.04.02 03:32:29.491136 [ 7614 ] {} TablesLoader: Parsed metadata of 0 tables in 1 databases in 4.3662e-05 sec Stdout:2025.04.02 03:32:29.491142 [ 7614 ] {} ReferentialDeps: No tables Stdout:2025.04.02 03:32:29.491150 [ 7614 ] {} LoadingDeps: No tables Stdout:2025.04.02 03:32:29.491175 [ 7614 ] {} loadMetadata: Start asynchronous loading of databases Stdout:2025.04.02 03:32:29.491182 [ 7614 ] {} AsyncLoader: Schedule load job 'startup Ordinary database default' into BackgrndStartup Stdout:2025.04.02 03:32:29.491186 [ 7614 ] {} AsyncLoader: Schedule load job 'startup Atomic database default' into BackgrndStartup Stdout:2025.04.02 03:32:29.491190 [ 7614 ] {} AsyncLoader: Change current priority: none -> 2 Stdout:2025.04.02 03:32:29.491193 [ 7614 ] {} AsyncLoader: Spawn loader worker #1 in BackgrndStartup Stdout:2025.04.02 03:32:29.491234 [ 7614 ] {} UserDefinedSQLObjectsLoaderFromDisk: Loading user defined objects from /var/lib/clickhouse/user_defined/ Stdout:2025.04.02 03:32:29.491246 [ 7614 ] {} UserDefinedSQLObjectsLoaderFromDisk: The directory for user defined objects (/var/lib/clickhouse/user_defined/) does not exist: nothing to load Stdout:2025.04.02 03:32:29.491242 [ 8298 ] {} AsyncLoader: Execute load job 'startup Ordinary database default' in BackgrndStartup Stdout:2025.04.02 03:32:29.491276 [ 8298 ] {} AsyncLoader: Finish load job 'startup Ordinary database default' with status OK Stdout:2025.04.02 03:32:29.491284 [ 7614 ] {} WorkloadEntityDiskStorage: Loading workload entities from /var/lib/clickhouse/workload/ Stdout:2025.04.02 03:32:29.491300 [ 7614 ] {} WorkloadEntityDiskStorage: The directory for workload entities (/var/lib/clickhouse/workload/) does not exist: nothing to load Stdout:2025.04.02 03:32:29.491286 [ 8298 ] {} AsyncLoader: Spawn loader worker #2 in BackgrndStartup Stdout:2025.04.02 03:32:29.491349 [ 8298 ] {} AsyncLoader: Execute load job 'startup Atomic database default' in BackgrndStartup Stdout:2025.04.02 03:32:29.491355 [ 8298 ] {} AsyncLoader: Finish load job 'startup Atomic database default' with status OK Stdout:2025.04.02 03:32:29.491359 [ 8298 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:29.491369 [ 8289 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:29.491384 [ 8289 ] {} AsyncLoader: Change current priority: 2 -> none Stdout:2025.04.02 03:32:29.491819 [ 7614 ] {} ZooKeeperClient: Adding ZooKeeper host zoo1:2181 (172.16.8.2:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:29.492164 [ 7614 ] {} ZooKeeperClient: Adding ZooKeeper host zoo3:2181 (172.16.8.3:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:29.492431 [ 7614 ] {} ZooKeeperClient: Adding ZooKeeper host zoo2:2181 (172.16.8.4:2181), az: UNKNOWN, priority: 0 Stdout:2025.04.02 03:32:29.493818 [ 7614 ] {} ZooKeeperClient: Connected to ZooKeeper at 172.16.8.2:2181 with session_id 15 Stdout:2025.04.02 03:32:29.495043 [ 7614 ] {} ZooKeeperClient: Keeper feature flag FILTERED_LIST: enabled Stdout:2025.04.02 03:32:29.495069 [ 7614 ] {} ZooKeeperClient: Keeper feature flag MULTI_READ: enabled Stdout:2025.04.02 03:32:29.495073 [ 7614 ] {} ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: enabled Stdout:2025.04.02 03:32:29.495076 [ 7614 ] {} ZooKeeperClient: Keeper feature flag CREATE_IF_NOT_EXISTS: enabled Stdout:2025.04.02 03:32:29.495079 [ 7614 ] {} ZooKeeperClient: Keeper feature flag REMOVE_RECURSIVE: enabled Stdout:2025.04.02 03:32:29.495085 [ 7614 ] {} ZooKeeper: Initialized, hosts: zoo1:2181,zoo2:2181,zoo3:2181 Stdout:2025.04.02 03:32:29.498589 [ 7614 ] {} Application: Loaded metadata. Stdout:2025.04.02 03:32:29.498723 [ 7614 ] {} bool DB::(anonymous namespace)::checkPermissionsImpl(): Code: 412. DB::Exception: Can't receive Netlink response: error -2. (NETLINK_ERROR) (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:29.498749 [ 7614 ] {} Application: Tasks stats provider: procfs Stdout:2025.04.02 03:32:29.498864 [ 7614 ] {} DatabaseAtomic (system): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:29.498873 [ 7614 ] {} DatabaseAtomic (system): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:29.499232 [ 7614 ] {} MySQLHandlerFactory: Failed to create SSL context. SSL will be disabled. Error: Poco::Exception. Code: 1000, e.code() = 0, SSL Exception: Configuration error: no certificate file has been specified (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:29.499285 [ 7614 ] {} MySQLHandlerFactory: Failed to read RSA key pair from server certificate. Error: Code: 139. DB::Exception: Certificate file is not set. (NO_ELEMENTS_IN_CONFIG) (version 25.2.2.96963.altinityantalya.96963 (official build)) Stdout:2025.04.02 03:32:29.499292 [ 7614 ] {} MySQLHandlerFactory: Generating new RSA key pair. Stdout:2025.04.02 03:32:29.531815 [ 7614 ] {} CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. Stdout:2025.04.02 03:32:29.531852 [ 7614 ] {} CertificateReloader: One of paths is empty. Cannot apply new configuration for certificates. Fill all paths and try again. Stdout:2025.04.02 03:32:29.535522 [ 7614 ] {} AsyncLoader: Schedule load job 'startup ddl worker' into BackgrndStartup Stdout:2025.04.02 03:32:29.535543 [ 7614 ] {} AsyncLoader: Change current priority: none -> 2 Stdout:2025.04.02 03:32:29.535551 [ 7614 ] {} AsyncLoader: Spawn loader worker #1 in BackgrndStartup Stdout:2025.04.02 03:32:29.535658 [ 7614 ] {} Application: Listening for http://0.0.0.0:8123 Stdout:2025.04.02 03:32:29.535692 [ 8307 ] {} AsyncLoader: Execute load job 'startup ddl worker' in BackgrndStartup Stdout:2025.04.02 03:32:29.535734 [ 8307 ] {} AsyncLoader: Finish load job 'startup ddl worker' with status OK Stdout:2025.04.02 03:32:29.535741 [ 8307 ] {} AsyncLoader: Stop worker in BackgrndStartup Stdout:2025.04.02 03:32:29.535745 [ 8307 ] {} AsyncLoader: Change current priority: 2 -> none Stdout:2025.04.02 03:32:29.535755 [ 8307 ] {} DDLWorker: Starting DDLWorker thread Stdout:2025.04.02 03:32:29.535759 [ 8307 ] {} DDLWorker: Initializing DDLWorker thread Stdout:2025.04.02 03:32:29.535876 [ 8295 ] {} DDLWorker: Started DDLWorker cleanup thread Stdout:2025.04.02 03:32:29.535901 [ 7614 ] {} Application: Listening for native protocol (tcp): 0.0.0.0:9000 Stdout:2025.04.02 03:32:29.536011 [ 7614 ] {} Application: Listening for MySQL compatibility protocol: 0.0.0.0:9004 Stdout:2025.04.02 03:32:29.536081 [ 7614 ] {} Application: Listening for PostgreSQL compatibility protocol: 0.0.0.0:9005 Stdout:2025.04.02 03:32:29.536092 [ 7614 ] {} Application: Ready for connections. Stdout:2025.04.02 03:32:29.541116 [ 8307 ] {} DDLWorker: Initialized DDLWorker thread Stdout:2025.04.02 03:32:29.541155 [ 8295 ] {} DDLWorker: Cleaning queue Stdout:2025.04.02 03:32:29.541551 [ 8307 ] {} DDLWorker: Scheduling tasks Stdout:2025.04.02 03:32:29.541564 [ 8307 ] {} DDLWorker: Don't have unfinished tasks after restarting Stdout:2025.04.02 03:32:29.541729 [ 8307 ] {} DDLWorker: scheduleTasks: initialized=true, size_before_filtering=0, queue_size=0, entries=none..none, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=none Stdout:2025.04.02 03:32:29.541741 [ 8307 ] {} DDLWorker: No tasks to schedule Stdout:2025.04.02 03:32:29.541748 [ 8307 ] {} DDLWorker: Waiting for queue updates Stdout:2025.04.02 03:32:30.203620 [ 7626 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63448 Stdout:2025.04.02 03:32:30.203704 [ 7626 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:30.203733 [ 7626 ] {} TCP-Session-8fec209b-73ec-4993-a075-51e07e816a1b: Authenticating user 'default' from 172.16.8.1:63448 Stdout:2025.04.02 03:32:30.203776 [ 7626 ] {} TCP-Session-8fec209b-73ec-4993-a075-51e07e816a1b: 8fec209b-73ec-4993-a075-51e07e816a1b Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:30.203784 [ 7626 ] {} TCP-Session-8fec209b-73ec-4993-a075-51e07e816a1b: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:30.203894 [ 7626 ] {} ContextAccess (default): Settings: readonly = 0, allow_ddl = true, allow_introspection_functions = false Stdout:2025.04.02 03:32:30.203944 [ 7626 ] {} ContextAccess (default): List of all grants: GRANT CHECK, SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, ROLE ADMIN, CREATE ROW POLICY, ALTER ROW POLICY, DROP ROW POLICY, CREATE QUOTA, ALTER QUOTA, DROP QUOTA, CREATE SETTINGS PROFILE, ALTER SETTINGS PROFILE, DROP SETTINGS PROFILE, ALLOW SQL SECURITY NONE, SHOW ACCESS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* WITH GRANT OPTION, GRANT TABLE ENGINE ON * WITH GRANT OPTION, GRANT CREATE USER, ALTER USER, DROP USER, CREATE ROLE, ALTER ROLE, DROP ROLE, SET DEFINER ON * WITH GRANT OPTION, GRANT ALTER NAMED COLLECTION, CREATE NAMED COLLECTION, DROP NAMED COLLECTION, SHOW NAMED COLLECTIONS, NAMED COLLECTION ON * WITH GRANT OPTION Stdout:2025.04.02 03:32:30.203966 [ 7626 ] {} ContextAccess (default): List of all grants including implicit: GRANT CHECK, SHOW, SELECT, INSERT, ALTER, CREATE, DROP, UNDROP TABLE, TRUNCATE, OPTIMIZE, BACKUP, KILL QUERY, KILL TRANSACTION, MOVE PARTITION BETWEEN SHARDS, ROLE ADMIN, CREATE ROW POLICY, ALTER ROW POLICY, DROP ROW POLICY, CREATE QUOTA, ALTER QUOTA, DROP QUOTA, CREATE SETTINGS PROFILE, ALTER SETTINGS PROFILE, DROP SETTINGS PROFILE, ALLOW SQL SECURITY NONE, SHOW ACCESS, SYSTEM, dictGet, displaySecretsInShowAndSelect, INTROSPECTION, SOURCES, CLUSTER ON *.* WITH GRANT OPTION, GRANT TABLE ENGINE ON * WITH GRANT OPTION, GRANT CREATE USER, ALTER USER, DROP USER, CREATE ROLE, ALTER ROLE, DROP ROLE, SET DEFINER ON * WITH GRANT OPTION, GRANT ALTER NAMED COLLECTION, CREATE NAMED COLLECTION, DROP NAMED COLLECTION, SHOW NAMED COLLECTIONS, NAMED COLLECTION ON * WITH GRANT OPTION Stdout:2025.04.02 03:32:30.213919 [ 7626 ] {} TCP-Session-8fec209b-73ec-4993-a075-51e07e816a1b: 8fec209b-73ec-4993-a075-51e07e816a1b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:30.214255 [ 7626 ] {8f2b9cca-d139-4716-8411-33af75c280d1} executeQuery: (from 172.16.8.1:63448) (query 1, line 1) select 20 (stage: Complete) Stdout:2025.04.02 03:32:30.215271 [ 7626 ] {8f2b9cca-d139-4716-8411-33af75c280d1} Planner: Query to stage Complete Stdout:2025.04.02 03:32:30.215402 [ 7626 ] {8f2b9cca-d139-4716-8411-33af75c280d1} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:30.216091 [ 7626 ] {8f2b9cca-d139-4716-8411-33af75c280d1} executeQuery: Read 1 rows, 1.00 B in 0.001818 sec., 550.0550055005501 rows/sec., 550.06 B/sec. Stdout:2025.04.02 03:32:30.216200 [ 7626 ] {8f2b9cca-d139-4716-8411-33af75c280d1} TCPHandler: Processed in 0.002482636 sec. Stdout:2025.04.02 03:32:30.226818 [ 7626 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:30.226842 [ 7626 ] {} TCP-Session-8fec209b-73ec-4993-a075-51e07e816a1b: 8fec209b-73ec-4993-a075-51e07e816a1b Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.272466 [ 7626 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63462 Stdout:2025.04.02 03:32:32.272521 [ 7626 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:32.272547 [ 7626 ] {} TCP-Session-77623f76-f894-40b9-b53f-540fa7322557: Authenticating user 'default' from 172.16.8.1:63462 Stdout:2025.04.02 03:32:32.272573 [ 7626 ] {} TCP-Session-77623f76-f894-40b9-b53f-540fa7322557: 77623f76-f894-40b9-b53f-540fa7322557 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.272581 [ 7626 ] {} TCP-Session-77623f76-f894-40b9-b53f-540fa7322557: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.281561 [ 7626 ] {} TCP-Session-77623f76-f894-40b9-b53f-540fa7322557: 77623f76-f894-40b9-b53f-540fa7322557 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:32.281814 [ 7626 ] {4f7448e2-da01-4048-a774-a69d9f78792e} executeQuery: (from 172.16.8.1:63462) (query 1, line 2) CREATE TABLE test_remove_volume_from_policy ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' (stage: Complete) Stdout:2025.04.02 03:32:32.282178 [ 7626 ] {4f7448e2-da01-4048-a774-a69d9f78792e} IInterpreterUnionOrSelectQuery: The new analyzer is enabled, but the old interpreter is used. It can be a bug, please report it. Will disable 'allow_experimental_analyzer' setting (for query: SELECT min(d), max(d), count() SETTINGS aggregate_functions_null_for_empty = false, transform_null_in = false, legacy_column_name_of_tuple_literal = false) Stdout:2025.04.02 03:32:32.282815 [ 7626 ] {4f7448e2-da01-4048-a774-a69d9f78792e} default.test_remove_volume_from_policy (9488d414-7958-4d1e-837b-06f2c3deb860): Loading data parts Stdout:2025.04.02 03:32:32.283248 [ 7626 ] {4f7448e2-da01-4048-a774-a69d9f78792e} default.test_remove_volume_from_policy (9488d414-7958-4d1e-837b-06f2c3deb860): There are no data parts Stdout:2025.04.02 03:32:32.283313 [ 7626 ] {4f7448e2-da01-4048-a774-a69d9f78792e} AsyncLoader: Prioritize load job 'startup Atomic database default': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:32.283320 [ 7626 ] {4f7448e2-da01-4048-a774-a69d9f78792e} AsyncLoader: Prioritize load job 'startup Ordinary database default': BackgrndStartup -> ForegroundLoad Stdout:2025.04.02 03:32:32.284964 [ 7626 ] {4f7448e2-da01-4048-a774-a69d9f78792e} DatabaseAtomic (default): There are 0 detached tables. Start searching non used tables. Stdout:2025.04.02 03:32:32.284976 [ 7626 ] {4f7448e2-da01-4048-a774-a69d9f78792e} DatabaseAtomic (default): Found 0 non used tables in detached tables. Stdout:2025.04.02 03:32:32.285236 [ 7640 ] {} default.test_remove_volume_from_policy (9488d414-7958-4d1e-837b-06f2c3deb860): Didn't start merge: There are no parts that can be merged. (Collector returned empty ranges set) Stdout:2025.04.02 03:32:32.285311 [ 7626 ] {4f7448e2-da01-4048-a774-a69d9f78792e} TCPHandler: Processed in 0.003869616 sec. Stdout:2025.04.02 03:32:32.285584 [ 7626 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:32.285598 [ 7626 ] {} TCP-Session-77623f76-f894-40b9-b53f-540fa7322557: 77623f76-f894-40b9-b53f-540fa7322557 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.342567 [ 7626 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63468 Stdout:2025.04.02 03:32:32.342632 [ 7626 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:32.342654 [ 7626 ] {} TCP-Session-5bb125b0-f4b9-4da1-8ed1-f8440d233ec4: Authenticating user 'default' from 172.16.8.1:63468 Stdout:2025.04.02 03:32:32.342685 [ 7626 ] {} TCP-Session-5bb125b0-f4b9-4da1-8ed1-f8440d233ec4: 5bb125b0-f4b9-4da1-8ed1-f8440d233ec4 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.342700 [ 7626 ] {} TCP-Session-5bb125b0-f4b9-4da1-8ed1-f8440d233ec4: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.358144 [ 7626 ] {} TCP-Session-5bb125b0-f4b9-4da1-8ed1-f8440d233ec4: 5bb125b0-f4b9-4da1-8ed1-f8440d233ec4 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:32.358412 [ 7626 ] {a3198099-ccb6-46b2-9d39-2c6175445cce} executeQuery: (from 172.16.8.1:63468) (query 1, line 1) SELECT volume_name FROM system.storage_policies WHERE policy_name = 'test_remove_volume_from_policy_cool_policy' (stage: Complete) Stdout:2025.04.02 03:32:32.358741 [ 7626 ] {a3198099-ccb6-46b2-9d39-2c6175445cce} Planner: Query to stage Complete Stdout:2025.04.02 03:32:32.358927 [ 7626 ] {a3198099-ccb6-46b2-9d39-2c6175445cce} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:32.359870 [ 7626 ] {a3198099-ccb6-46b2-9d39-2c6175445cce} executeQuery: Read 17 rows, 1.70 KiB in 0.001515 sec., 11221.12211221122 rows/sec., 1.09 MiB/sec. Stdout:2025.04.02 03:32:32.360006 [ 7626 ] {a3198099-ccb6-46b2-9d39-2c6175445cce} TCPHandler: Processed in 0.001997766 sec. Stdout:2025.04.02 03:32:32.373036 [ 7626 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:32.373080 [ 7626 ] {} TCP-Session-5bb125b0-f4b9-4da1-8ed1-f8440d233ec4: 5bb125b0-f4b9-4da1-8ed1-f8440d233ec4 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.413284 [ 7626 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63472 Stdout:2025.04.02 03:32:32.413336 [ 7626 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:32.413357 [ 7626 ] {} TCP-Session-b4578690-abdc-4f1a-aafa-e669b60b1b02: Authenticating user 'default' from 172.16.8.1:63472 Stdout:2025.04.02 03:32:32.413392 [ 7626 ] {} TCP-Session-b4578690-abdc-4f1a-aafa-e669b60b1b02: b4578690-abdc-4f1a-aafa-e669b60b1b02 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.413408 [ 7626 ] {} TCP-Session-b4578690-abdc-4f1a-aafa-e669b60b1b02: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.422203 [ 7626 ] {} TCP-Session-b4578690-abdc-4f1a-aafa-e669b60b1b02: b4578690-abdc-4f1a-aafa-e669b60b1b02 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:32.422409 [ 7626 ] {1f3ff049-f2ea-4c85-94b5-0b9863895fe8} executeQuery: (from 172.16.8.1:63472) (query 1, line 1) SELECT disks FROM system.storage_policies WHERE policy_name = 'test_remove_volume_from_policy_cool_policy' (stage: Complete) Stdout:2025.04.02 03:32:32.422630 [ 7626 ] {1f3ff049-f2ea-4c85-94b5-0b9863895fe8} Planner: Query to stage Complete Stdout:2025.04.02 03:32:32.422760 [ 7626 ] {1f3ff049-f2ea-4c85-94b5-0b9863895fe8} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:32.423456 [ 7626 ] {1f3ff049-f2ea-4c85-94b5-0b9863895fe8} executeQuery: Read 17 rows, 1.70 KiB in 0.001081 sec., 15726.17946345976 rows/sec., 1.53 MiB/sec. Stdout:2025.04.02 03:32:32.423547 [ 7626 ] {1f3ff049-f2ea-4c85-94b5-0b9863895fe8} TCPHandler: Processed in 0.001442616 sec. Stdout:2025.04.02 03:32:32.436731 [ 7626 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:32.436760 [ 7626 ] {} TCP-Session-b4578690-abdc-4f1a-aafa-e669b60b1b02: b4578690-abdc-4f1a-aafa-e669b60b1b02 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.481983 [ 7626 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63478 Stdout:2025.04.02 03:32:32.482028 [ 7626 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:32.482043 [ 7626 ] {} TCP-Session-f4e055d5-5dfc-4915-8291-fa006e97c96e: Authenticating user 'default' from 172.16.8.1:63478 Stdout:2025.04.02 03:32:32.482078 [ 7626 ] {} TCP-Session-f4e055d5-5dfc-4915-8291-fa006e97c96e: f4e055d5-5dfc-4915-8291-fa006e97c96e Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.482084 [ 7626 ] {} TCP-Session-f4e055d5-5dfc-4915-8291-fa006e97c96e: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.490540 [ 7626 ] {} TCP-Session-f4e055d5-5dfc-4915-8291-fa006e97c96e: f4e055d5-5dfc-4915-8291-fa006e97c96e Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:32.490719 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} executeQuery: (from 172.16.8.1:63478) (query 1, line 1) SYSTEM RELOAD CONFIG (stage: Complete) Stdout:2025.04.02 03:32:32.492394 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigReloader: Loading config '/etc/clickhouse-server/config.xml' Stdout:2025.04.02 03:32:32.492407 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigProcessor: Processing configuration file '/etc/clickhouse-server/config.xml'. Stdout:2025.04.02 03:32:32.493275 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/macros.xml'. Stdout:2025.04.02 03:32:32.493358 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/conf.d/zookeeper_config.xml'. Stdout:2025.04.02 03:32:32.493429 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_enable_keeper_async_replication.xml'. Stdout:2025.04.02 03:32:32.493480 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/0_common_instance_config.xml'. Stdout:2025.04.02 03:32:32.493637 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/logs_config.xml'. Stdout:2025.04.02 03:32:32.493716 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigProcessor: Merging configuration file '/etc/clickhouse-server/config.d/storage_configuration.xml'. Stdout:2025.04.02 03:32:32.498061 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'. Stdout:2025.04.02 03:32:32.498151 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performing update on configuration Stdout:2025.04.02 03:32:32.504358 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} Application: Changed setting 'max_server_memory_usage' to 27.54 GiB (30.60 GiB available memory * 0.90 max_server_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:32.504376 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.30 GiB (30.60 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) Stdout:2025.04.02 03:32:32.504380 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} Application: Merges and mutations memory limit is set to 15.30 GiB Stdout:2025.04.02 03:32:32.505287 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} Application: Setting max_remote_read_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:32.505297 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} Application: Setting max_remote_write_network_bandwidth_for_server was set to 0 Stdout:2025.04.02 03:32:32.505303 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} Application: ConcurrencyControl limit is set to 32 CPU slots with 'fair_round_robin' scheduler Stdout:2025.04.02 03:32:32.505314 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} MergeTreeBackgroundExecutor: Loaded new threads count (16) and max tasks count (32) for MergeMutateExecutor Stdout:2025.04.02 03:32:32.505321 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for MoveExecutor Stdout:2025.04.02 03:32:32.505325 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} MergeTreeBackgroundExecutor: Loaded new threads count (16) and max tasks count (16) for FetchExecutor Stdout:2025.04.02 03:32:32.505329 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} MergeTreeBackgroundExecutor: Loaded new threads count (8) and max tasks count (8) for CommonExecutor Stdout:2025.04.02 03:32:32.505857 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicy (small_jbod_with_external): Storage policy small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:32.505901 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicySelector: Storage policy `small_jbod_with_external` loaded Stdout:2025.04.02 03:32:32.506019 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicy (one_more_small_jbod_with_external): Storage policy one_more_small_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:32.506024 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicySelector: Storage policy `one_more_small_jbod_with_external` loaded Stdout:2025.04.02 03:32:32.506161 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicy (jbods_with_external): Storage policy jbods_with_external created, total volumes 2 Stdout:2025.04.02 03:32:32.506167 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicySelector: Storage policy `jbods_with_external` loaded Stdout:2025.04.02 03:32:32.506279 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicy (moving_jbod_with_external): Storage policy moving_jbod_with_external created, total volumes 2 Stdout:2025.04.02 03:32:32.506283 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicySelector: Storage policy `moving_jbod_with_external` loaded Stdout:2025.04.02 03:32:32.506349 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StorageConfiguration: Volume `small` max_data_part_size is too low (2.00 MiB < 8.00 MiB) Stdout:2025.04.02 03:32:32.506529 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicy (default_disk_with_external): Storage policy default_disk_with_external created, total volumes 2 Stdout:2025.04.02 03:32:32.506537 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicySelector: Storage policy `default_disk_with_external` loaded Stdout:2025.04.02 03:32:32.506695 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StorageConfiguration: Volume `special_warning_small_volume` max_data_part_size is too low (1.00 KiB < 8.00 MiB) Stdout:2025.04.02 03:32:32.506775 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicy (special_warning_policy): Storage policy special_warning_policy created, total volumes 4 Stdout:2025.04.02 03:32:32.506779 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicySelector: Storage policy `special_warning_policy` loaded Stdout:2025.04.02 03:32:32.506844 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicy (cool_policy): Storage policy cool_policy created, total volumes 1 Stdout:2025.04.02 03:32:32.506848 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicySelector: Storage policy `cool_policy` loaded Stdout:2025.04.02 03:32:32.506862 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} StoragePolicy (default): Storage policy default created, total volumes 1 Stdout:2025.04.02 03:32:32.506946 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} Context: An error has occurred while reloading storage policies, storage policies were not applied: Storage policy `test_remove_volume_from_policy_cool_policy` is missing in new configuration Stdout:2025.04.02 03:32:32.507081 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration Stdout:2025.04.02 03:32:32.508748 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigReloader: Loading config '/etc/clickhouse-server/users.xml' Stdout:2025.04.02 03:32:32.509379 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigProcessor: Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'. Stdout:2025.04.02 03:32:32.509398 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performing update on configuration Stdout:2025.04.02 03:32:32.509853 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performed update on configuration Stdout:2025.04.02 03:32:32.510250 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} MemoryTracker: Query peak memory usage: 10.28 MiB. Stdout:2025.04.02 03:32:32.510294 [ 7626 ] {33f81a79-4579-4c09-a369-269b869b44fa} TCPHandler: Processed in 0.019839862 sec. Stdout:2025.04.02 03:32:32.512352 [ 7626 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:32.512368 [ 7626 ] {} TCP-Session-f4e055d5-5dfc-4915-8291-fa006e97c96e: f4e055d5-5dfc-4915-8291-fa006e97c96e Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.546464 [ 7626 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63488 Stdout:2025.04.02 03:32:32.546509 [ 7626 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:32.546525 [ 7626 ] {} TCP-Session-17c48985-4c0d-4f9f-a54f-4068eb169c18: Authenticating user 'default' from 172.16.8.1:63488 Stdout:2025.04.02 03:32:32.546554 [ 7626 ] {} TCP-Session-17c48985-4c0d-4f9f-a54f-4068eb169c18: 17c48985-4c0d-4f9f-a54f-4068eb169c18 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.546561 [ 7626 ] {} TCP-Session-17c48985-4c0d-4f9f-a54f-4068eb169c18: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.556224 [ 7626 ] {} TCP-Session-17c48985-4c0d-4f9f-a54f-4068eb169c18: 17c48985-4c0d-4f9f-a54f-4068eb169c18 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:32.556443 [ 7626 ] {a1f181c2-6aba-4d8e-9c4a-d5ce52bd255a} executeQuery: (from 172.16.8.1:63488) (query 1, line 1) SELECT volume_name FROM system.storage_policies WHERE policy_name = 'test_remove_volume_from_policy_cool_policy' (stage: Complete) Stdout:2025.04.02 03:32:32.556700 [ 7626 ] {a1f181c2-6aba-4d8e-9c4a-d5ce52bd255a} Planner: Query to stage Complete Stdout:2025.04.02 03:32:32.556833 [ 7626 ] {a1f181c2-6aba-4d8e-9c4a-d5ce52bd255a} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:32.557512 [ 7626 ] {a1f181c2-6aba-4d8e-9c4a-d5ce52bd255a} executeQuery: Read 17 rows, 1.70 KiB in 0.001103 sec., 15412.51133272892 rows/sec., 1.50 MiB/sec. Stdout:2025.04.02 03:32:32.557594 [ 7626 ] {a1f181c2-6aba-4d8e-9c4a-d5ce52bd255a} TCPHandler: Processed in 0.001503629 sec. Stdout:2025.04.02 03:32:32.568021 [ 7626 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:32.568047 [ 7626 ] {} TCP-Session-17c48985-4c0d-4f9f-a54f-4068eb169c18: 17c48985-4c0d-4f9f-a54f-4068eb169c18 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.610998 [ 7626 ] {} TCPHandlerFactory: TCP Request. Address: 172.16.8.1:63498 Stdout:2025.04.02 03:32:32.611042 [ 7626 ] {} TCPHandler: Connected ClickHouse client version 25.2.0, revision: 54476, user: default. Stdout:2025.04.02 03:32:32.611067 [ 7626 ] {} TCP-Session-92f6fb0a-3ffc-4583-9099-b9f1c0f4577a: Authenticating user 'default' from 172.16.8.1:63498 Stdout:2025.04.02 03:32:32.611090 [ 7626 ] {} TCP-Session-92f6fb0a-3ffc-4583-9099-b9f1c0f4577a: 92f6fb0a-3ffc-4583-9099-b9f1c0f4577a Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.611096 [ 7626 ] {} TCP-Session-92f6fb0a-3ffc-4583-9099-b9f1c0f4577a: Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db Stdout:2025.04.02 03:32:32.620307 [ 7626 ] {} TCP-Session-92f6fb0a-3ffc-4583-9099-b9f1c0f4577a: 92f6fb0a-3ffc-4583-9099-b9f1c0f4577a Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default Stdout:2025.04.02 03:32:32.620496 [ 7626 ] {eddbd3af-de2e-4624-b040-e558a9441e84} executeQuery: (from 172.16.8.1:63498) (query 1, line 1) SELECT disks FROM system.storage_policies WHERE policy_name = 'test_remove_volume_from_policy_cool_policy' (stage: Complete) Stdout:2025.04.02 03:32:32.620672 [ 7626 ] {eddbd3af-de2e-4624-b040-e558a9441e84} Planner: Query to stage Complete Stdout:2025.04.02 03:32:32.620792 [ 7626 ] {eddbd3af-de2e-4624-b040-e558a9441e84} Planner: Query from stage FetchColumns to stage Complete Stdout:2025.04.02 03:32:32.621453 [ 7626 ] {eddbd3af-de2e-4624-b040-e558a9441e84} executeQuery: Read 17 rows, 1.70 KiB in 0.000991 sec., 17154.389505549952 rows/sec., 1.67 MiB/sec. Stdout:2025.04.02 03:32:32.621547 [ 7626 ] {eddbd3af-de2e-4624-b040-e558a9441e84} TCPHandler: Processed in 0.001333701 sec. Stdout:2025.04.02 03:32:32.630778 [ 7626 ] {} TCPHandler: Done processing connection. Stdout:2025.04.02 03:32:32.630798 [ 7626 ] {} TCP-Session-92f6fb0a-3ffc-4583-9099-b9f1c0f4577a: 92f6fb0a-3ffc-4583-9099-b9f1c0f4577a Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db Executing query DROP TABLE IF EXISTS test_remove_volume_from_policy on node1 [gw2] PASSED test_reloading_storage_configuration/test.py::test_remove_volume_from_policy Executing query ALTER TABLE test_rename_with_parallel_merges RENAME COLUMN num2 to foo2 on node1 test_reloading_storage_configuration/test.py::test_update_disk Executing query ALTER TABLE test_rename_with_parallel_merges RENAME COLUMN foo2 to foo3 on node2 Executing query ALTER TABLE test_rename_with_parallel_merges RENAME COLUMN foo3 to num2 on node3 Executing query OPTIMIZE TABLE test_rename_with_parallel_merges FINAL on node1 Executing query OPTIMIZE TABLE test_rename_with_parallel_merges FINAL on node2 Executing query OPTIMIZE TABLE test_rename_with_parallel_merges FINAL on node3 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps -C clickhouse] thread 0: insert for 2000-01-01: [4, 10, 10, 4, 4, 7, 8, 6, 2, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout: PID TTY TIME CMD Stdout: 7614 ? 00:00:00 clickhouse run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -9 clickhouse'] Command:[docker exec -u root roottestreloadingstorageconfiguration-gw2-node1-1 bash -c pkill -9 clickhouse] Executing query ALTER TABLE test_rename_with_parallel_merges RENAME COLUMN foo3 to num2 on node3 Executing query ALTER TABLE test_rename_with_parallel_merges RENAME COLUMN foo2 to foo3 on node2 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:91036794 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:7614 Stdout:273013120 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:210004138 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 1: insert for 2000-01-02: [5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:277135 Executing query SELECT count() FROM system.replicated_fetches on node2 Executing query SELECT count() FROM system.replication_queue WHERE postpone_reason like '%fetches have already throttled%' on node2 thread 0: insert for 2000-01-01: [4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Stderr: node Pulling Stderr: node Pulled ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/.env --project-name roottestreloadquerymaskingrules-gw7 --file /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/node/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/.env --project-name roottestreloadquerymaskingrules-gw7 --file /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/node/docker-compose.yml up -d --no-recreate] Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query SELECT * FROM `test_database`.`postgresql_replica_5` LIMIT 1 FORMAT Null on instance thread 1: insert for 2000-01-02: [9, 3, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_5' on instance thread 0: insert for 2000-01-01: [5, 8, 3, 4, 4, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `postgres_database`.`postgresql_replica_5` order by key; on instance run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Executing query select * from `test_database`.`postgresql_replica_5` order by key; on instance Stdout:91037982 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:276121292 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:212648531 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 1: insert for 2000-01-02: [6, 10, 10, 4, 3, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:284967 Executing query SELECT count() FROM system.replicated_fetches on node2 thread 0: insert for 2000-01-01: [2, 6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT count() FROM system.replication_queue WHERE postpone_reason like '%fetches have already throttled%' on node2 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stderr: Network roottestreloadquerymaskingrules-gw7_default Creating Stderr: Network roottestreloadquerymaskingrules-gw7_default Created Stderr: Container roottestreloadquerymaskingrules-gw7-node-1 Creating Stderr: Container roottestreloadquerymaskingrules-gw7-node-1 Created Stderr: Container roottestreloadquerymaskingrules-gw7-node-1 Starting Stderr: Container roottestreloadquerymaskingrules-gw7-node-1 Started ClickHouse instance created get_instance_ip instance_name=node http://localhost:None "GET /v1.46/containers/roottestreloadquerymaskingrules-gw7-node-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node http://localhost:None "GET /v1.46/containers/roottestreloadquerymaskingrules-gw7-node-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node, ip: 172.16.3.2... http://localhost:None "GET /v1.46/containers/roottestreloadquerymaskingrules-gw7-node-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/3b16391fb6cf99d8de698cb8d66e82faed02c4eebb69d20961661ba8744f9625/json HTTP/1.1" 200 None No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreloadingstorageconfiguration-gw2-node1-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/bffd49fd764bac1891d23d4741d6c6dbe2e1efe49c88bfbf398fd7a4c3fffe4d/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/bffd49fd764bac1891d23d4741d6c6dbe2e1efe49c88bfbf398fd7a4c3fffe4d/json HTTP/1.1" 200 586 http://localhost:None "GET /v1.46/containers/3b16391fb6cf99d8de698cb8d66e82faed02c4eebb69d20961661ba8744f9625/json HTTP/1.1" 200 None Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Stopped Executing query SELECT COUNT() FROM test_table on node http://localhost:None "GET /v1.46/containers/3b16391fb6cf99d8de698cb8d66e82faed02c4eebb69d20961661ba8744f9625/json HTTP/1.1" 200 None Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [1, 9, 10, 5, 10, 3, 1, 1, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT COUNT() FROM test_table on node thread 0: insert for 2000-01-01: [6, 7, 10, 7, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 http://localhost:None "GET /v1.46/containers/3b16391fb6cf99d8de698cb8d66e82faed02c4eebb69d20961661ba8744f9625/json HTTP/1.1" 200 None Starting zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadzookeeper-gw5 --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml start zoo2] http://localhost:None "GET /v1.46/containers/3b16391fb6cf99d8de698cb8d66e82faed02c4eebb69d20961661ba8744f9625/json HTTP/1.1" 200 None run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] http://localhost:None "GET /v1.46/containers/3b16391fb6cf99d8de698cb8d66e82faed02c4eebb69d20961661ba8744f9625/json HTTP/1.1" 200 None ClickHouse node started run container_id:roottestreloadquerymaskingrules-gw7-node-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVlcnlfbG9nPgogICAgICAgIDxkYXRhYmFzZT5zeXN0ZW08L2RhdGFiYXNlPgogICAgICAgIDx0YWJsZT5xdWVyeV9sb2c8L3RhYmxlPgogICAgICAgIDxwYXJ0aXRpb25fYnk+dG9ZWVlZTU0oZXZlbnRfZGF0ZSk8L3BhcnRpdGlvbl9ieT4KICAgICAgICA8Zmx1c2hfaW50ZXJ2YWxfbWlsbGlzZWNvbmRzPjc1MDA8L2ZsdXNoX2ludGVydmFsX21pbGxpc2Vjb25kcz4KICAgICAgICA8bWF4X3NpemVfcm93cz4xMDQ4NTc2PC9tYXhfc2l6ZV9yb3dzPgogICAgICAgIDxyZXNlcnZlZF9zaXplX3Jvd3M+ODE5MjwvcmVzZXJ2ZWRfc2l6ZV9yb3dzPgogICAgICAgIDxidWZmZXJfc2l6ZV9yb3dzX2ZsdXNoX3RocmVzaG9sZD41MjQyODg8L2J1ZmZlcl9zaXplX3Jvd3NfZmx1c2hfdGhyZXNob2xkPgogICAgICAgIDxmbHVzaF9vbl9jcmFzaD5mYWxzZTwvZmx1c2hfb25fY3Jhc2g+CiAgICA8L3F1ZXJ5X2xvZz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/config.d/z.xml'] Command:[docker exec roottestreloadquerymaskingrules-gw7-node-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVlcnlfbG9nPgogICAgICAgIDxkYXRhYmFzZT5zeXN0ZW08L2RhdGFiYXNlPgogICAgICAgIDx0YWJsZT5xdWVyeV9sb2c8L3RhYmxlPgogICAgICAgIDxwYXJ0aXRpb25fYnk+dG9ZWVlZTU0oZXZlbnRfZGF0ZSk8L3BhcnRpdGlvbl9ieT4KICAgICAgICA8Zmx1c2hfaW50ZXJ2YWxfbWlsbGlzZWNvbmRzPjc1MDA8L2ZsdXNoX2ludGVydmFsX21pbGxpc2Vjb25kcz4KICAgICAgICA8bWF4X3NpemVfcm93cz4xMDQ4NTc2PC9tYXhfc2l6ZV9yb3dzPgogICAgICAgIDxyZXNlcnZlZF9zaXplX3Jvd3M+ODE5MjwvcmVzZXJ2ZWRfc2l6ZV9yb3dzPgogICAgICAgIDxidWZmZXJfc2l6ZV9yb3dzX2ZsdXNoX3RocmVzaG9sZD41MjQyODg8L2J1ZmZlcl9zaXplX3Jvd3NfZmx1c2hfdGhyZXNob2xkPgogICAgICAgIDxmbHVzaF9vbl9jcmFzaD5mYWxzZTwvZmx1c2hfb25fY3Jhc2g+CiAgICA8L3F1ZXJ5X2xvZz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/config.d/z.xml] Stdout:91039038 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query SYSTEM RELOAD CONFIG on node thread 1: insert for 2000-01-02: [7, 2, 8, 9, 7, 1, 2, 9, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:279372166 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Starting Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Started Starting zookeeper node: zoo3 Command:[docker compose --project-name roottestreloadzookeeper-gw5 --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml start zoo3] Stdout:215901586 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] thread 0: insert for 2000-01-01: [7, 7, 5, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT 'TOPSECRET.TOPSECRET' on node Stdout:292892 Executing query SELECT count() FROM system.replicated_fetches on node2 run container_id:roottestreloadquerymaskingrules-gw7-node-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "SELECT \'TOPSECRET.TOPSECRET\'" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] Command:[docker exec roottestreloadquerymaskingrules-gw7-node-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "SELECT 'TOPSECRET.TOPSECRET'" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Executing query SELECT count() FROM system.replication_queue WHERE postpone_reason like '%fetches have already throttled%' on node2 Executing query select * from `test_database`.`postgresql_replica_5` order by key; on instance Stdout:/var/log/clickhouse-server/clickhouse-server.log:2025.04.02 03:32:34.824710 [ 10 ] {a837e050-f96c-4dc2-ac13-c942db5f7455} executeQuery: (from 172.16.3.1:61256) (query 1, line 1) SELECT 'TOPSECRET.TOPSECRET' (stage: Complete) Stdout:/var/log/clickhouse-server/clickhouse-server.log:2025.04.02 03:32:34.824745 [ 10 ] {a837e050-f96c-4dc2-ac13-c942db5f7455} CancellationChecker: Did not add the task because the timeout is 0. Query: SELECT 'TOPSECRET.TOPSECRET' run container_id:roottestreloadquerymaskingrules-gw7-node-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "SELECT \'\\[hidden\\]\'" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] Command:[docker exec roottestreloadquerymaskingrules-gw7-node-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "SELECT '\[hidden\]'" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] run container_id:roottestreloadquerymaskingrules-gw7-node-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadquerymaskingrules-gw7-node-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 1: insert for 2000-01-02: [5, 6, 4, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:1 run container_id:roottestreloadquerymaskingrules-gw7-node-1 detach:False nothrow:False cmd: ['bash', '-c', 'kill -HUP 1'] Command:[docker exec -u root roottestreloadquerymaskingrules-gw7-node-1 bash -c kill -HUP 1] Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Starting Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Started get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.5.2, port:2181, use_ssl:False Connecting to 172.16.5.2(172.16.5.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused run container_id:roottestreloadquerymaskingrules-gw7-node-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVlcnlfbG9nPgogICAgICAgIDxkYXRhYmFzZT5zeXN0ZW08L2RhdGFiYXNlPgogICAgICAgIDx0YWJsZT5xdWVyeV9sb2c8L3RhYmxlPgogICAgICAgIDxwYXJ0aXRpb25fYnk+dG9ZWVlZTU0oZXZlbnRfZGF0ZSk8L3BhcnRpdGlvbl9ieT4KICAgICAgICA8Zmx1c2hfaW50ZXJ2YWxfbWlsbGlzZWNvbmRzPjc1MDA8L2ZsdXNoX2ludGVydmFsX21pbGxpc2Vjb25kcz4KICAgICAgICA8bWF4X3NpemVfcm93cz4xMDQ4NTc2PC9tYXhfc2l6ZV9yb3dzPgogICAgICAgIDxyZXNlcnZlZF9zaXplX3Jvd3M+ODE5MjwvcmVzZXJ2ZWRfc2l6ZV9yb3dzPgogICAgICAgIDxidWZmZXJfc2l6ZV9yb3dzX2ZsdXNoX3RocmVzaG9sZD41MjQyODg8L2J1ZmZlcl9zaXplX3Jvd3NfZmx1c2hfdGhyZXNob2xkPgogICAgICAgIDxmbHVzaF9vbl9jcmFzaD5mYWxzZTwvZmx1c2hfb25fY3Jhc2g+CiAgICA8L3F1ZXJ5X2xvZz4KCiAgICA8cXVlcnlfbWFza2luZ19ydWxlcz4KICAgICAgICA8cnVsZT4KICAgICAgICAgICAgPHJlZ2V4cD5UT1BTRUNSRVQuVE9QU0VDUkVUPC9yZWdleHA+CiAgICAgICAgICAgIDxyZXBsYWNlPltoaWRkZW5dPC9yZXBsYWNlPgogICAgICAgIDwvcnVsZT4KICAgIDwvcXVlcnlfbWFza2luZ19ydWxlcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/config.d/z.xml'] Command:[docker exec roottestreloadquerymaskingrules-gw7-node-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z.xml) && echo PGNsaWNraG91c2U+CiAgICA8cXVlcnlfbG9nPgogICAgICAgIDxkYXRhYmFzZT5zeXN0ZW08L2RhdGFiYXNlPgogICAgICAgIDx0YWJsZT5xdWVyeV9sb2c8L3RhYmxlPgogICAgICAgIDxwYXJ0aXRpb25fYnk+dG9ZWVlZTU0oZXZlbnRfZGF0ZSk8L3BhcnRpdGlvbl9ieT4KICAgICAgICA8Zmx1c2hfaW50ZXJ2YWxfbWlsbGlzZWNvbmRzPjc1MDA8L2ZsdXNoX2ludGVydmFsX21pbGxpc2Vjb25kcz4KICAgICAgICA8bWF4X3NpemVfcm93cz4xMDQ4NTc2PC9tYXhfc2l6ZV9yb3dzPgogICAgICAgIDxyZXNlcnZlZF9zaXplX3Jvd3M+ODE5MjwvcmVzZXJ2ZWRfc2l6ZV9yb3dzPgogICAgICAgIDxidWZmZXJfc2l6ZV9yb3dzX2ZsdXNoX3RocmVzaG9sZD41MjQyODg8L2J1ZmZlcl9zaXplX3Jvd3NfZmx1c2hfdGhyZXNob2xkPgogICAgICAgIDxmbHVzaF9vbl9jcmFzaD5mYWxzZTwvZmx1c2hfb25fY3Jhc2g+CiAgICA8L3F1ZXJ5X2xvZz4KCiAgICA8cXVlcnlfbWFza2luZ19ydWxlcz4KICAgICAgICA8cnVsZT4KICAgICAgICAgICAgPHJlZ2V4cD5UT1BTRUNSRVQuVE9QU0VDUkVUPC9yZWdleHA+CiAgICAgICAgICAgIDxyZXBsYWNlPltoaWRkZW5dPC9yZXBsYWNlPgogICAgICAgIDwvcnVsZT4KICAgIDwvcXVlcnlfbWFza2luZ19ydWxlcz4KPC9jbGlja2hvdXNlPgo= | base64 --decode > /etc/clickhouse-server/config.d/z.xml] Executing query SYSTEM RELOAD CONFIG on node Connecting to 172.16.5.2(172.16.5.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 0: insert for 2000-01-01: [1, 1, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SELECT 'TOPSECRET.TOPSECRET' on node Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Stdout:8391 Clickhouse process running. run container_id:roottestreloadingstorageconfiguration-gw2-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadingstorageconfiguration-gw2-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestreloadquerymaskingrules-gw7-node-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "SELECT \'\\[hidden\\]\'" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] Command:[docker exec roottestreloadquerymaskingrules-gw7-node-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "SELECT '\[hidden\]'" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] Stdout:8391 Executing query select 20 on node1 Stdout:/var/log/clickhouse-server/clickhouse-server.log:2025.04.02 03:32:35.189232 [ 10 ] {3a79ea57-6c58-4070-a9b9-be9589852b5c} executeQuery: (from 172.16.3.1:63266) (query 1, line 1) SELECT '[hidden]' (stage: Complete) Stdout:/var/log/clickhouse-server/clickhouse-server.log:2025.04.02 03:32:35.189259 [ 10 ] {3a79ea57-6c58-4070-a9b9-be9589852b5c} CancellationChecker: Did not add the task because the timeout is 0. Query: SELECT '[hidden]' run container_id:roottestreloadquerymaskingrules-gw7-node-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "SELECT \'TOPSECRET.TOPSECRET\'" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true'] Command:[docker exec roottestreloadquerymaskingrules-gw7-node-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -aH "SELECT 'TOPSECRET.TOPSECRET'" /var/log/clickhouse-server/clickhouse-server.log | ( [ -z "" ] && cat || grep -v "$" ) || true] thread 1: insert for 2000-01-02: [4, 10, 10, 1, 4, 10, 3, 9, 5, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreloadquerymaskingrules-gw7-node-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreloadquerymaskingrules-gw7-node-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Connecting to 172.16.5.2(172.16.5.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Stdout:1 run container_id:roottestreloadquerymaskingrules-gw7-node-1 detach:False nothrow:False cmd: ['bash', '-c', 'kill -HUP 1'] Command:[docker exec -u root roottestreloadquerymaskingrules-gw7-node-1 bash -c kill -HUP 1] Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/.env --project-name roottestreloadquerymaskingrules-gw7 --file /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/node/docker-compose.yml stop --timeout 20] [gw7] PASSED test_reload_query_masking_rules/test.py::test_reload_query_masking_rules run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 0: insert for 2000-01-01: [5, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:91040226 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:282529262 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:219060875 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:300948 Executing query SELECT count() FROM system.replicated_fetches on node2 thread 1: insert for 2000-01-02: [3, 3, 10, 7, 2, 10, 10, 7, 10, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT count() FROM system.replication_queue WHERE postpone_reason like '%fetches have already throttled%' on node2 Executing query ALTER TABLE test_rename_with_parallel_merges RENAME COLUMN num2 to foo2 on node1 Executing query OPTIMIZE TABLE test_rename_with_parallel_merges FINAL on node3 thread 0: insert for 2000-01-01: [4, 1, 2, 8, 1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.5.2(172.16.5.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 1: insert for 2000-01-02: [6, 7, 8, 2, 4, 8, 7, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `test_database`.`postgresql_replica_5` order by key; on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_abrupt_server_restart_while_heavy_replication thread 0: insert for 2000-01-01: [6, 7, 8, 3, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] thread 1: insert for 2000-01-02: [2, 7, 1, 3, 3, 3, 2, 10, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:91041282 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:284569802 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:221206470 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:309105 Executing query SELECT count() FROM system.replicated_fetches on node2 Executing query SELECT count() FROM system.replication_queue WHERE postpone_reason like '%fetches have already throttled%' on node2 Executing query OPTIMIZE TABLE test_rename_with_parallel_merges FINAL on node1 thread 1: insert for 2000-01-02: [10, 10, 8, 3, 3, 9, 2, 1, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance thread 1: insert for 2000-01-02: [4, 6, 6, 7, 7, 5, 8, 7, 9, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_changing_replica_identity_value Executing query INSERT INTO postgres_database.postgresql_replica SELECT 50 + number, number from numbers(50) on instance run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Stdout:91042926 run container_id:roottestreplicatedfetchesbandwidth-gw6-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node1-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Executing query DROP DATABASE IF EXISTS `test_database` on instance thread 0: insert for 2000-01-01: [5, 6, 4, 2, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:287750672 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { sub(/^.*:/, "") ; print $1 } else { print $2 } }' /proc/net/dev] Connecting to 172.16.5.2(172.16.5.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Stdout:224286457 run container_id:roottestreplicatedfetchesbandwidth-gw6-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'awk "/^ *eth0:/"\' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }\' /proc/net/dev'] Command:[docker exec roottestreplicatedfetchesbandwidth-gw6-node2-1 bash -c awk "/^ *eth0:/"' { if ($1 ~ /.*:[0-9][0-9]*/) { print $9 } else { print $10 } }' /proc/net/dev] Stdout:317147 Executing query SELECT count() FROM system.replicated_fetches on node2 Executing query OPTIMIZE TABLE test_rename_with_parallel_merges FINAL on node2 Executing query SHOW DATABASES on instance Executing query SELECT count() FROM system.replication_queue WHERE postpone_reason like '%fetches have already throttled%' on node2 Executing query INSERT INTO postgres_database.postgresql_replica SELECT 100 + number, number from numbers(50) on instance thread 1: insert for 2000-01-02: [7, 5, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query CREATE TABLE test_update_disk ( d UInt64 ) ENGINE = MergeTree() ORDER BY d SETTINGS storage_policy='jbods_with_external' on node1 Executing query SELECT * FROM `test_database`.`postgresql_replica` LIMIT 1 FORMAT Null on instance Executing query SELECT path, keep_free_space FROM system.disks where name = 'jbod2' on node1 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica' on instance Executing query SYSTEM RELOAD CONFIG on node1 thread 0: insert for 2000-01-01: [3, 10, 8, 6, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `postgres_database`.`postgresql_replica` order by key; on instance Executing query SELECT path, keep_free_space FROM system.disks where name = 'jbod2' on node1 thread 1: insert for 2000-01-02: [4, 7, 9, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP TABLE IF EXISTS test_update_disk on node1 Executing query select * from `test_database`.`postgresql_replica` order by key; on instance Command:[docker compose --env-file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/.env --project-name roottestreloadingstorageconfiguration-gw2 --file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node2/docker-compose.yml stop --timeout 20] [gw2] PASSED test_reloading_storage_configuration/test.py::test_update_disk Executing query SELECT * FROM `test_database`.`postgresql_replica` LIMIT 1 FORMAT Null on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query SYSTEM SYNC REPLICA should_execute_table on node2 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica' on instance thread 1: insert for 2000-01-02: [9, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [7, 7, 1, 10, 2, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `postgres_database`.`postgresql_replica` order by key; on instance Executing query select * from `test_database`.`postgresql_replica` order by key; on instance thread 1: insert for 2000-01-02: [3, 4, 3, 6, 8, 8, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestreloadquerymaskingrules-gw7-node-1 Stopping Stderr: Container roottestreloadquerymaskingrules-gw7-node-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/.env --project-name roottestreloadquerymaskingrules-gw7 --file /ClickHouse/tests/integration/test_reload_query_masking_rules/_instances-0-gw7/node/docker-compose.yml down --volumes] thread 0: insert for 2000-01-01: [4, 8, 7, 1, 8, 3, 2, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query ALTER TABLE test_rename_with_parallel_merges RENAME COLUMN foo3 to num2 on node1 Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Stopped Stopping zookeeper node: zoo3 Command:[docker compose --project-name roottestremovestalemovingparts-gw1 --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop zoo3] thread 1: insert for 2000-01-02: [1, 1, 1, 2, 2, 8, 8, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 0: insert for 2000-01-01: [5, 9, 8, 3, 4, 1, 10, 10, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Executing query select * from `test_database`.`postgresql_replica` order by key; on instance thread 1: insert for 2000-01-02: [3, 4, 8, 5, 1, 1, 3, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestreloadquerymaskingrules-gw7-node-1 Stopping Stderr: Container roottestreloadquerymaskingrules-gw7-node-1 Stopped Stderr: Container roottestreloadquerymaskingrules-gw7-node-1 Removing Stderr: Container roottestreloadquerymaskingrules-gw7-node-1 Removed Stderr: Network roottestreloadquerymaskingrules-gw7_default Removing Stderr: Network roottestreloadquerymaskingrules-gw7_default Removed Cleanup called Docker networks for project roottestreloadquerymaskingrules-gw7 are NETWORK ID NAME DRIVER SCOPE Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_changing_replica_identity_value Docker containers for project roottestreloadquerymaskingrules-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreloadquerymaskingrules-gw7 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreloadquerymaskingrules-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreloadquerymaskingrules-gw7 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:9 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 9 test_replicated_engine_arguments/test.py::test_replicated_engine_with_arguments Running tests in /ClickHouse/tests/integration/test_replicated_engine_arguments/test.py Cluster start called. is_up=False Docker networks for project roottestreplicatedenginearguments-gw7 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedenginearguments-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedenginearguments-gw7 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestreplicatedenginearguments-gw7 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedenginearguments-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedenginearguments-gw7 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedenginearguments-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] thread 1: insert for 2000-01-02: [1, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Unstopped containers: {} No running containers for project: roottestreplicatedenginearguments-gw7 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:9 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 9 Setup directory for instance: node1 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_engine_arguments/configs/remote_servers.xml'] to /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/node1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/node1/database Setup logs dir /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/node1/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Stderr: Container roottestreloadingstorageconfiguration-gw2-node2-1 Stopping Stderr: Container roottestreloadingstorageconfiguration-gw2-node1-1 Stopping Stderr: Container roottestreloadingstorageconfiguration-gw2-node2-1 Stopped Stderr: Container roottestreloadingstorageconfiguration-gw2-node1-1 Stopped Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Stopping Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Stopping Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Stopping Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Stopped Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Stopped Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/.env --project-name roottestreplicatedenginearguments-gw7 --file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml pull] Executing query ALTER TABLE test_rename_with_parallel_merges RENAME COLUMN foo2 to num2 on node1 Connecting to 172.16.5.2(172.16.5.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Command:[bash -c [ -f /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/.env --project-name roottestreloadingstorageconfiguration-gw2 --file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_reloading_storage_configuration/_instances-0-gw2/node2/docker-compose.yml down --volumes] Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Executing query SELECT count() FROM test_rename_with_parallel_merges WHERE num2 % 1000 > 0 on node1 Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Closing connection to 172.16.2.2:2281 Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.2.3, port:2281, use_ssl:True Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SELECT count() FROM test_rename_with_parallel_merges WHERE num2 % 1000 > 0 on node2 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Closing connection to 172.16.2.3:2281 Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.2.4, port:2281, use_ssl:True Connecting to 172.16.2.4(172.16.2.4):2281, use_ssl: True Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) thread 0: insert for 2000-01-01: [4, 3, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Closing connection to 172.16.2.4:2281 Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ['zoo1', 'zoo2', 'zoo3'] run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo -n > /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c echo -n > /var/log/clickhouse-server/clickhouse-server.log] thread 1: insert for 2000-01-02: [1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo -n > /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c echo -n > /var/log/clickhouse-server/clickhouse-server.log] run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF\n\n \n \n /etc/clickhouse-server/config.d/second_client.crt\n /etc/clickhouse-server/config.d/second_client.key\n true\n true\n sslv2,sslv3\n true\n none\n \n RejectCertificateHandler\n \n \n \n\nEOF'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF /etc/clickhouse-server/config.d/second_client.crt /etc/clickhouse-server/config.d/second_client.key true true sslv2,sslv3 true none RejectCertificateHandler EOF] run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'touch /etc/clickhouse-server/config.d/ssl_conf.xml'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c touch /etc/clickhouse-server/config.d/ssl_conf.xml] Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF\n\n \n \n /etc/clickhouse-server/config.d/second_client.crt\n /etc/clickhouse-server/config.d/second_client.key\n true\n true\n sslv2,sslv3\n true\n none\n \n RejectCertificateHandler\n \n \n \n\nEOF'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c cat > /etc/clickhouse-server/config.d/ssl_conf.xml << EOF /etc/clickhouse-server/config.d/second_client.crt /etc/clickhouse-server/config.d/second_client.key true true sslv2,sslv3 true none RejectCertificateHandler EOF] run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'touch /etc/clickhouse-server/config.d/ssl_conf.xml'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c touch /etc/clickhouse-server/config.d/ssl_conf.xml] test_postgresql_replica_database_engine_1/test.py::test_clickhouse_restart Executing query INSERT INTO `postgres_database`.postgresql_replica_0 SELECT number, number from numbers(50) on instance Executing query SELECT count() FROM test_rename_with_parallel_merges WHERE num2 % 1000 > 0 on node3 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_merges SYNC on node1 Executing query INSERT INTO `postgres_database`.postgresql_replica_1 SELECT number, number from numbers(50) on instance Stderr: Container roottestreloadingstorageconfiguration-gw2-node2-1 Stopping Stderr: Container roottestreloadingstorageconfiguration-gw2-node1-1 Stopping Stderr: Container roottestreloadingstorageconfiguration-gw2-node2-1 Stopped Stderr: Container roottestreloadingstorageconfiguration-gw2-node2-1 Removing Stderr: Container roottestreloadingstorageconfiguration-gw2-node1-1 Stopped Stderr: Container roottestreloadingstorageconfiguration-gw2-node1-1 Removing Stderr: Container roottestreloadingstorageconfiguration-gw2-node2-1 Removed Stderr: Container roottestreloadingstorageconfiguration-gw2-node1-1 Removed Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Stopping Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Stopping Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Stopping Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Stopped Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Removing Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Stopped Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Removing Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Stopped Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Removing Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo1-1 Removed Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo2-1 Removed Stderr: Container roottestreloadingstorageconfiguration-gw2-zoo3-1 Removed Stderr: Network roottestreloadingstorageconfiguration-gw2_default Removing Stderr: Network roottestreloadingstorageconfiguration-gw2_default Removed Cleanup called Docker networks for project roottestreloadingstorageconfiguration-gw2 are NETWORK ID NAME DRIVER SCOPE thread 0: insert for 2000-01-01: [2, 5, 4, 1, 3, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Docker containers for project roottestreloadingstorageconfiguration-gw2 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Executing query INSERT INTO `postgres_database`.postgresql_replica_2 SELECT number, number from numbers(50) on instance Docker volumes for project roottestreloadingstorageconfiguration-gw2 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreloadingstorageconfiguration-gw2-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreloadingstorageconfiguration-gw2 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Executing query INSERT INTO `postgres_database`.postgresql_replica_3 SELECT number, number from numbers(50) on instance Stdout:9 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 9 thread 1: insert for 2000-01-02: [7, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO `postgres_database`.postgresql_replica_4 SELECT number, number from numbers(50) on instance Executing query DROP TABLE IF EXISTS test_rename_with_parallel_merges SYNC on node2 Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance thread 0: insert for 2000-01-01: [10, 8, 4, 2, 6, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SHOW DATABASES on instance Executing query DROP TABLE IF EXISTS test_rename_with_parallel_merges SYNC on node3 Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance thread 1: insert for 2000-01-02: [7, 7, 8, 6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance thread 0: insert for 2000-01-01: [6, 6, 2, 1, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query DROP TABLE IF EXISTS test_rename_with_parallel_merges SYNC on node4 Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance thread 1: insert for 2000-01-02: [4, 6, 7, 10, 2, 7, 9, 9, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance [gw3] PASSED test_rename_column/test.py::test_rename_with_parallel_merges test_rename_column/test.py::test_rename_with_parallel_select Executing query SELECT value FROM system.build_options WHERE name = 'CXX_FLAGS' on node1 thread 0: insert for 2000-01-01: [10, 7, 9, 2, 4, 4, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance Executing query DROP TABLE IF EXISTS test_rename_with_parallel_select SYNC on node1 Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance Executing query DROP TABLE IF EXISTS test_rename_with_parallel_select SYNC on node2 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance Executing query DROP TABLE IF EXISTS test_rename_with_parallel_select SYNC on node3 Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query DROP TABLE IF EXISTS test_rename_with_parallel_select SYNC on node4 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance Executing query CREATE TABLE test_rename_with_parallel_select ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_select', 'node1') ORDER BY num PARTITION BY num % 100 on node1 thread 0: insert for 2000-01-01: [7, 7, 2, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance thread 1: insert for 2000-01-02: [7, 1, 6, 6, 10, 1, 1, 3, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query CREATE TABLE test_rename_with_parallel_select ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_select', 'node2') ORDER BY num PARTITION BY num % 100 on node2 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query CREATE TABLE test_rename_with_parallel_select ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_select', 'node3') ORDER BY num PARTITION BY num % 100 on node3 Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Executing query CREATE TABLE test_rename_with_parallel_select ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_select', 'node4') ORDER BY num PARTITION BY num % 100 on node4 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_select (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(1000) on node1 Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance thread 0: insert for 2000-01-01: [3, 1, 10, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance thread 1: insert for 2000-01-02: [3, 6, 3, 9, 6, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO postgres_database.postgresql_replica_0 SELECT 50 + number, 0 from numbers(50000) on instance Executing query INSERT INTO postgres_database.postgresql_replica_1 SELECT 50 + number, 1 from numbers(50000) on instance thread 0: insert for 2000-01-01: [6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [6, 7, 7, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO postgres_database.postgresql_replica_2 SELECT 50 + number, 2 from numbers(50000) on instance Executing query INSERT INTO postgres_database.postgresql_replica_3 SELECT 50 + number, 3 from numbers(50000) on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE num2 % 1000 > 0 on node1 thread 0: insert for 2000-01-01: [5, 5, 3, 2, 9, 9, 4, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query INSERT INTO postgres_database.postgresql_replica_4 SELECT 50 + number, 4 from numbers(50000) on instance Executing query SELECT count() FROM test_rename_with_parallel_select WHERE num2 % 1000 > 0 on node2 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE num2 % 1000 > 0 on node3 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps -C clickhouse] Stdout: PID TTY TIME CMD Stdout: 795 ? 00:00:03 clickhouse run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill clickhouse'] Command:[docker exec -u root roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c pkill clickhouse] Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo2 to foo3 on node2 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo3 to num2 on node3 Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SELECT count() FROM test_rename_with_parallel_select WHERE foo3 % 1000 > 0 on node1 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE num2 % 1000 > 0 on node2 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE foo2 % 1000 > 0 on node3 thread 0: insert for 2000-01-01: [5, 1, 4, 1, 6, 7, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [7, 4, 2, 9, 8, 1, 2, 10, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:795 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE foo2 % 1000 > 0 on node3 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE foo3 % 1000 > 0 on node1 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo3 to num2 on node3 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE num2 % 1000 > 0 on node2 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo3 to num2 on node3 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE foo3 % 1000 > 0 on node1 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo3 to num2 on node3 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE foo2 % 1000 > 0 on node3 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE num2 % 1000 > 0 on node2 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE foo3 % 1000 > 0 on node1 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo3 to num2 on node3 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE num2 % 1000 > 0 on node2 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE foo3 % 1000 > 0 on node1 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE foo2 % 1000 > 0 on node3 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE num2 % 1000 > 0 on node2 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE foo2 % 1000 > 0 on node3 thread 1: insert for 2000-01-02: [2, 1, 2, 8, 10, 7, 9, 6, 6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [6, 10, 2, 5, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [2, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [7, 4, 10, 9, 10, 10, 4, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN num2 to foo2 on node1 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN num2 to foo2 on node1 thread 1: insert for 2000-01-02: [6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:795 thread 0: insert for 2000-01-01: [6, 3, 7, 2, 9, 6, 5, 9, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [5, 2, 2, 3, 3, 5, 10, 10, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [6, 6, 8, 7, 4, 4, 3, 2, 10, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo2 to foo3 on node2 Connecting to 172.16.5.2(172.16.5.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['clickhouse', 'keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo2 to foo3 on node2 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo2 to foo3 on node2 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.5.3, port:2181, use_ssl:False thread 1: insert for 2000-01-02: [10, 1, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.5.3(172.16.5.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper', 'clickhouse'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo2 to foo3 on node2 Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo3 to num2 on node1 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ['zoo2', 'zoo3'] Executing query SELECT COUNT() FROM test_table on node Executing query SELECT COUNT() FROM test_table on node thread 0: insert for 2000-01-01: [8, 1, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreloadzookeeper-gw5-node-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo \'\n\n \n \n zoo2\n 2181\n \n 2000\n \n\n\' > /etc/clickhouse-server/conf.d/zookeeper.xml'] Command:[docker exec roottestreloadzookeeper-gw5-node-1 bash -c echo ' zoo2 2181 2000 ' > /etc/clickhouse-server/conf.d/zookeeper.xml] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SYSTEM RELOAD CONFIG on node Stdout:795 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [4, 2, 7, 9, 9, 7, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [8, 1, 3, 10, 3, 1, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 run container_id:roottestreloadzookeeper-gw5-node-1 detach:False nothrow:False cmd: ['ss', '--resolve', '--tcp', '--no-header', 'state', 'ESTABLISHED', '( dport = 2181 or sport = 2181 )'] Command:[docker exec -u root --privileged roottestreloadzookeeper-gw5-node-1 ss --resolve --tcp --no-header state ESTABLISHED ( dport = 2181 or sport = 2181 )] thread 0: insert for 2000-01-01: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:0 0 node:51938 roottestreloadzookeeper-gw5-zoo2-1.roottestreloadzookeeper-gw5_default:2181 Executing query SELECT COUNT() FROM test_table on node run container_id:roottestreloadzookeeper-gw5-node-1 detach:False nothrow:False cmd: ['ss', '--resolve', '--tcp', '--no-header', 'state', 'ESTABLISHED', '( dport = 2181 or sport = 2181 )'] Command:[docker exec -u root --privileged roottestreloadzookeeper-gw5-node-1 ss --resolve --tcp --no-header state ESTABLISHED ( dport = 2181 or sport = 2181 )] Stdout:0 0 node:51938 roottestreloadzookeeper-gw5-zoo2-1.roottestreloadzookeeper-gw5_default:2181 Starting zookeeper node: zoo1 Command:[docker compose --project-name roottestreloadzookeeper-gw5 --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml start zoo1] Executing query ALTER TABLE test_rename_with_parallel_select RENAME COLUMN foo2 to num2 on node1 Executing query SELECT count() FROM test_rename_with_parallel_select WHERE num2 % 1000 > 0 on node1 thread 1: insert for 2000-01-02: [9, 6, 10, 5, 9, 7, 3, 3, 6, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_select SYNC on node1 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Starting Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Started Starting zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadzookeeper-gw5 --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml start zoo2] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Starting zookeeper node: zoo3 Command:[docker compose --project-name roottestreloadzookeeper-gw5 --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml start zoo3] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestpostgresqlreplicadatabaseengine1-gw0-instance-1/exec HTTP/1.1" 201 74 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_select SYNC on node2 http://localhost:None "POST /v1.46/exec/5ad546f739964c926614bee5d62113a9fef3dd61b19748a4ab81714afd15567d/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/5ad546f739964c926614bee5d62113a9fef3dd61b19748a4ab81714afd15567d/json HTTP/1.1" 200 586 get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.5.4, port:2181, use_ssl:False Connecting to 172.16.5.4(172.16.5.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT COUNT() FROM should_execute_table on node2 Connecting to 172.16.5.4(172.16.5.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['clickhouse', 'keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query DROP TABLE IF EXISTS test_rename_with_parallel_select SYNC on node3 thread 1: insert for 2000-01-02: [8, 3, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP TABLE IF EXISTS should_execute_table SYNC on node1 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.5.2, port:2181, use_ssl:False Connecting to 172.16.5.2(172.16.5.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['clickhouse', 'keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query DROP TABLE IF EXISTS test_rename_with_parallel_select SYNC on node4 Executing query DROP TABLE IF EXISTS should_execute_table SYNC on node2 thread 0: insert for 2000-01-01: [2, 2, 1, 5, 5, 5, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreloadzookeeper-gw5-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.5.3, port:2181, use_ssl:False Connecting to 172.16.5.3(172.16.5.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper', 'clickhouse'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/.env --project-name roottestreplicatedfetchesbandwidth-gw6 --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node3/docker-compose.yml stop --timeout 20] [gw6] PASSED test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch [gw3] PASSED test_rename_column/test.py::test_rename_with_parallel_select test_rename_column/test.py::test_rename_with_parallel_slow_insert Executing query SELECT value FROM system.build_options WHERE name = 'CXX_FLAGS' on node1 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ['zoo1', 'zoo2', 'zoo3'] run container_id:roottestreloadzookeeper-gw5-node-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo \'\n\n \n \n zoo1\n 2181\n \n \n zoo2\n 2181\n \n \n zoo3\n 2181\n \n 20000\n \n\n \' > /etc/clickhouse-server/conf.d/zookeeper.xml'] Command:[docker exec roottestreloadzookeeper-gw5-node-1 bash -c echo ' zoo1 2181 zoo2 2181 zoo3 2181 20000 ' > /etc/clickhouse-server/conf.d/zookeeper.xml] thread 1: insert for 2000-01-02: [1, 10, 4, 3, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SYSTEM RELOAD CONFIG on node Executing query DROP TABLE IF EXISTS test_rename_with_parallel_slow_insert SYNC on node1 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_slow_insert SYNC on node2 Executing query DROP TABLE test_table on node Executing query DROP TABLE IF EXISTS test_rename_with_parallel_slow_insert SYNC on node3 Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --project-name roottestreloadzookeeper-gw5 --file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop --timeout 20] [gw5] PASSED test_reload_zookeeper/test.py::test_reload_zookeeper Executing query DROP TABLE IF EXISTS test_rename_with_parallel_slow_insert SYNC on node4 thread 0: insert for 2000-01-01: [8, 3, 10, 2, 2, 7, 5, 6, 2, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query CREATE TABLE test_rename_with_parallel_slow_insert ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_slow_insert', 'node1') ORDER BY num PARTITION BY num % 100 on node1 thread 1: insert for 2000-01-02: [6, 9, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query CREATE TABLE test_rename_with_parallel_slow_insert ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_slow_insert', 'node2') ORDER BY num PARTITION BY num % 100 on node2 Executing query CREATE TABLE test_rename_with_parallel_slow_insert ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_slow_insert', 'node3') ORDER BY num PARTITION BY num % 100 on node3 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query CREATE TABLE test_rename_with_parallel_slow_insert ( num UInt32, num2 UInt32 DEFAULT num + 1 ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_slow_insert', 'node4') ORDER BY num PARTITION BY num % 100 on node4 Stdout:1587 Clickhouse process running. run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 0: insert for 2000-01-01: [8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:1587 Executing query select 20 on instance Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_slow_insert (num,num2) SELECT number + 0 AS num, number + 1 + 0 AS num2 FROM numbers_mt(1000) on node1 Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance thread 1: insert for 2000-01-02: [4, 6, 4, 7, 9, 6, 2, 4, 5, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance thread 0: insert for 2000-01-01: [6, 10, 5, 6, 5, 9, 7, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [10, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance thread 0: insert for 2000-01-01: [4, 4, 3, 7, 9, 8, 1, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance thread 1: insert for 2000-01-02: [2, 9, 8, 4, 3, 9, 4] Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_slow_insert (num, num2) SELECT number + sleepEachRow(0.001) AS num, number + 1 AS num2 FROM numbers_mt(10000) SETTINGS function_sleep_max_microseconds_per_block = 0 on node1 Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_slow_insert (num, num2) SELECT number + sleepEachRow(0.001) AS num, number + 1 AS num2 FROM numbers_mt(10000) SETTINGS function_sleep_max_microseconds_per_block = 0 on node1 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance thread 0: insert for 2000-01-01: [4, 1, 6, 10, 9, 4, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [10, 3, 9, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Executing query ALTER TABLE test_rename_with_parallel_slow_insert RENAME COLUMN num2 to foo2 on node1 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 0: insert for 2000-01-01: [8, 7, 6, 5, 2, 8, 4, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance thread 1: insert for 2000-01-02: [10, 3, 10, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_clickhouse_restart Executing query DROP DATABASE IF EXISTS "postgres_database" on instance thread 1: insert for 2000-01-02: [7, 2, 1, 6, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [10, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance thread 1: insert for 2000-01-02: [7, 3, 10, 4, 7, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 grep in log called Reloaded certificate (/etc/clickhouse-server/config.d/second_client.crt) run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', '[ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -a "Reloaded certificate (/etc/clickhouse-server/config.d/second_client.crt)" /var/log/clickhouse-server/clickhouse-server.log* || true'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c [ -f /var/log/clickhouse-server/clickhouse-server.log ] && zgrep -a "Reloaded certificate (/etc/clickhouse-server/config.d/second_client.crt)" /var/log/clickhouse-server/clickhouse-server.log* || true] thread 0: insert for 2000-01-01: [7, 1, 1, 10, 9, 1, 2, 1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: zoo3 Skipped - Image is already being pulled by zoo2 Stderr: node1 Skipped - Image is already being pulled by zoo2 Stderr: zoo1 Skipped - Image is already being pulled by zoo2 Stderr: zoo2 Pulling Stderr: zoo2 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper1/log', '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper1/config', '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper1/coordination', '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper2/log', '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper2/config', '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper2/coordination', '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper3/log', '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper3/config', '/ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/keeper3/coordination'] Command:[docker compose --project-name roottestreplicatedenginearguments-gw7 --env-file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stdout:2025.04.02 03:32:40.366939 [ 670 ] {} CertificateReloader: Reloaded certificate (/etc/clickhouse-server/config.d/second_client.crt) and key (/etc/clickhouse-server/config.d/second_client.key). grep result 2025.04.02 03:32:40.366939 [ 670 ] {} CertificateReloader: Reloaded certificate (/etc/clickhouse-server/config.d/second_client.crt) and key (/etc/clickhouse-server/config.d/second_client.key). Stopping zookeeper node: zoo1 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo1] Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node1-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node3-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node2-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node3-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node1-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node2-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] test_postgresql_replica_database_engine_1/test.py::test_concurrent_transactions Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Executing query DROP DATABASE IF EXISTS `test_database` on instance Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/.env --project-name roottestreplicatedmergetrees3zerocopy-gw8 --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_s3_zero_copy/_instances-0-gw8/node3/docker-compose.yml down --volumes] Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance thread 0: insert for 2000-01-01: [5, 5, 8, 4, 7, 7, 3, 7, 8, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [8, 4, 3, 10, 8, 5, 5, 7, 3, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node3-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node1-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node2-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node3-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node3-1 Removing Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node1-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node1-1 Removing Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node2-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node2-1 Removing Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Removing Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node3-1 Removed Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node1-1 Removed Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-resolver-1 Removed Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Removing Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-node2-1 Removed Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Removing Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Removing Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Removing Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo3-1 Removed Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo2-1 Removed Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-zoo1-1 Removed Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-minio1-1 Removed Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Stopping Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Removing Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Stopped Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Removing Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy2-1 Removed Stderr: Container roottestreplicatedmergetrees3zerocopy-gw8-proxy1-1 Removed Stderr: Volume roottestreplicatedmergetrees3zerocopy-gw8_data1-1 Removing Stderr: Network roottestreplicatedmergetrees3zerocopy-gw8_default Removing Stderr: Volume roottestreplicatedmergetrees3zerocopy-gw8_data1-1 Removed Stderr: Network roottestreplicatedmergetrees3zerocopy-gw8_default Removed Cleanup called Docker networks for project roottestreplicatedmergetrees3zerocopy-gw8 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedmergetrees3zerocopy-gw8 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedmergetrees3zerocopy-gw8 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmergetrees3zerocopy-gw8-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicatedmergetrees3zerocopy-gw8 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopped Stopping zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo2] Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 thread 0: insert for 2000-01-01: [9, 9, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [1, 4, 10, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node1-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node2-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node3-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node2-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node1-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node3-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/.env --project-name roottestreplicatedfetchesbandwidth-gw6 --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_fetches_bandwidth/_instances-0-gw6/node3/docker-compose.yml down --volumes] thread 0: insert for 2000-01-01: [7, 2, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [10, 7, 6, 5, 3, 7, 9, 4, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 0: insert for 2000-01-01: [5, 1, 3, 3, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopped Stopping zookeeper node: zoo3 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml stop zoo3] Stderr:time="2025-04-02T03:32:50Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreplicatedenginearguments-gw7_default Creating Stderr: Network roottestreplicatedenginearguments-gw7_default Created Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Creating Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Creating Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Creating Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Created Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Created Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Created Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Starting Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Starting Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Starting Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Started Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Started Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Started Stderr:time="2025-04-02T03:32:52Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:32:52Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreplicatedenginearguments-gw7-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.3.4, port:2181, use_ssl:False Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node1-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node2-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node3-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node1-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node1-1 Removing Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node3-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node3-1 Removing Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node2-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node2-1 Removing Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node2-1 Removed Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node3-1 Removed Stderr: Container roottestreplicatedfetchesbandwidth-gw6-node1-1 Removed Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Stopping Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Removing Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Removing Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Stopped Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Removing Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo3-1 Removed Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo2-1 Removed Stderr: Container roottestreplicatedfetchesbandwidth-gw6-zoo1-1 Removed Stderr: Network roottestreplicatedfetchesbandwidth-gw6_default Removing Stderr: Network roottestreplicatedfetchesbandwidth-gw6_default Removed Cleanup called Docker networks for project roottestreplicatedfetchesbandwidth-gw6 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedfetchesbandwidth-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedfetchesbandwidth-gw6 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedfetchesbandwidth-gw6-.*-1$' --format '{{.ID}}:{{.Names}}'] Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Unstopped containers: {} No running containers for project: roottestreplicatedfetchesbandwidth-gw6 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] thread 1: insert for 2000-01-02: [9, 3, 9, 7, 6, 2, 4, 10, 9, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings Starting cluster... Running tests in /ClickHouse/tests/integration/test_replicated_merge_tree_config/test.py Cluster start called. is_up=False Docker networks for project roottestreplicatedmergetreeconfig-gw6 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedmergetreeconfig-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedmergetreeconfig-gw6 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestreplicatedmergetreeconfig-gw6 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedmergetreeconfig-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES thread 0: insert for 2000-01-01: [10, 6, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Docker volumes for project roottestreplicatedmergetreeconfig-gw6 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmergetreeconfig-gw6-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicatedmergetreeconfig-gw6 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 Setup directory for instance: node Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_merge_tree_config/configs/config.xml'] to /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/node/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/node/database Setup logs dir /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/node/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/.env --project-name roottestreplicatedmergetreeconfig-gw6 --file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml pull] thread 1: insert for 2000-01-02: [6, 5, 9, 1, 6, 3, 5, 4, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopped Starting zookeeper node: zoo1 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo1] thread 0: insert for 2000-01-01: [9, 10, 9, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [9, 6, 9, 1, 7, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Stopped Executing query SYSTEM STOP MOVES on ch1 thread 0: insert for 2000-01-01: [4, 3, 7, 8, 6, 4, 1, 3, 4, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SYSTEM DISABLE FAILPOINT stop_moving_part_before_swap_with_active on ch1 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Started Starting zookeeper node: zoo2 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo2] run container_id:roottestremovestalemovingparts-gw1-ch1-1 detach:False nothrow:False cmd: ['bash', '-c', 'ls /var/lib/clickhouse/disks/s3/store/8af/8aff6f37-927b-406a-9a36-fc430f60964d/moving'] Command:[docker exec roottestremovestalemovingparts-gw1-ch1-1 bash -c ls /var/lib/clickhouse/disks/s3/store/8af/8aff6f37-927b-406a-9a36-fc430f60964d/moving] Stderr: Container roottestreloadzookeeper-gw5-node-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-node-1 Stopped Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Stopped Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Stopped Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/.env --project-name roottestreloadzookeeper-gw5 --file /ClickHouse/tests/integration/test_reload_zookeeper/_instances-0-gw5/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml down --volumes] Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Starting zookeeper node: zoo1 Command:[docker compose --project-name roottestremovestalemovingparts-gw1 --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml start zoo1] Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [3, 9, 2, 10, 6, 7, 1, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [2, 1, 6, 2, 2, 3, 1, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT count() FROM postgres_database.postgresql_replica_0 on instance thread 1: insert for 2000-01-02: [5, 2, 6, 8, 2, 8, 5, 2, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [1, 1, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Started Starting zookeeper node: zoo2 Command:[docker compose --project-name roottestremovestalemovingparts-gw1 --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml start zoo2] Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Started Starting zookeeper node: zoo3 Command:[docker compose --project-name roottestreloadclientcertificate-gw4 --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml start zoo3] Executing query SELECT count() FROM (SELECT * FROM test_database.postgresql_replica_0) on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Stderr: Container roottestreloadzookeeper-gw5-node-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-node-1 Stopped Stderr: Container roottestreloadzookeeper-gw5-node-1 Removing Stderr: Container roottestreloadzookeeper-gw5-node-1 Removed Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Stopping Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Stopped Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Removing Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Stopped Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Removing Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Stopped Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Removing Stderr: Container roottestreloadzookeeper-gw5-zoo3-1 Removed Stderr: Container roottestreloadzookeeper-gw5-zoo2-1 Removed Stderr: Container roottestreloadzookeeper-gw5-zoo1-1 Removed Stderr: Network roottestreloadzookeeper-gw5_default Removing Stderr: Network roottestreloadzookeeper-gw5_default Removed Cleanup called Docker networks for project roottestreloadzookeeper-gw5 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreloadzookeeper-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreloadzookeeper-gw5 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreloadzookeeper-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreloadzookeeper-gw5 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:7 Command:[docker volume prune -f] thread 1: insert for 2000-01-02: [10, 7, 1, 6, 4, 3, 8, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [1, 1, 1] Stdout:Total reclaimed space: 0B Executing query INSERT INTO test_mutations FORMAT TSV on node5 Volumes pruned: 7 test_remote_prewhere/test.py::test_remote Running tests in /ClickHouse/tests/integration/test_remote_prewhere/test.py Cluster start called. is_up=False Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Started Starting zookeeper node: zoo3 Command:[docker compose --project-name roottestremovestalemovingparts-gw1 --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml start zoo3] Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance Docker networks for project roottestremoteprewhere-gw5 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestremoteprewhere-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestremoteprewhere-gw5 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestremoteprewhere-gw5 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestremoteprewhere-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Starting Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Started run container_id:roottestreloadclientcertificate-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo -n > /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadclientcertificate-gw4-node1-1 bash -c echo -n > /var/log/clickhouse-server/clickhouse-server.log] Docker volumes for project roottestremoteprewhere-gw5 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestremoteprewhere-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestremoteprewhere-gw5 Trying to prune unused networks... Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Trying to prune unused images... Command:[docker image prune -f] run container_id:roottestreloadclientcertificate-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'echo -n > /var/log/clickhouse-server/clickhouse-server.log'] Command:[docker exec roottestreloadclientcertificate-gw4-node2-1 bash -c echo -n > /var/log/clickhouse-server/clickhouse-server.log] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] get_instance_ip instance_name=zoo1 Stdout:7 Command:[docker volume prune -f] http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.2.2, port:2281, use_ssl:True Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Stdout:Total reclaimed space: 0B Volumes pruned: 7 Setup directory for instance: node1 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files [] to /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node1/database Setup logs dir /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node1/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup directory for instance: node2 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files [] to /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node2/configs/config.d Setup database dir /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node2/database Setup logs dir /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node2/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw'} stored in /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/.env --project-name roottestremoteprewhere-gw5 --file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node2/docker-compose.yml pull] Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Starting Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Started get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.7.2, port:2181, use_ssl:False Connecting to 172.16.7.2(172.16.7.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused thread 1: insert for 2000-01-02: [3, 9, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.7.2(172.16.7.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 0: insert for 2000-01-01: [10, 1, 5, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Connecting to 172.16.7.2(172.16.7.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 1: insert for 2000-01-02: [5, 4, 9, 3, 1, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused thread 0: insert for 2000-01-01: [2, 7, 9, 6, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.7.2(172.16.7.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 1: insert for 2000-01-02: [10, 1, 9, 4, 2, 9, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [8, 5, 7, 8, 2, 10, 7, 10, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused Executing query SELECT count() FROM postgres_database.postgresql_replica_1 on instance thread 1: insert for 2000-01-02: [9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT count() FROM (SELECT * FROM test_database.postgresql_replica_1) on instance Connecting to 172.16.7.2(172.16.7.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance thread 0: insert for 2000-01-01: [8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance thread 1: insert for 2000-01-02: [6, 6, 8, 3, 6, 5, 1, 9, 5, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 0: insert for 2000-01-01: [1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [8] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True Connection dropped: socket connection error: Connection refused thread 0: insert for 2000-01-01: [5, 3, 9, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [3, 3, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Connecting to 172.16.7.2(172.16.7.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance thread 0: insert for 2000-01-01: [3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [8, 9, 2, 4, 7, 8, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [3, 3, 6, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 1: insert for 2000-01-02: [1, 10, 2, 2, 6, 5, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node5 thread 0: insert for 2000-01-01: [9, 6, 4, 2, 1, 9, 7, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost thread 1: insert for 2000-01-02: [7] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance thread 0: insert for 2000-01-01: [3, 10, 5, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreplicatedenginearguments-gw7-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.3.3, port:2181, use_ssl:False Connecting to 172.16.3.3(172.16.3.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query SELECT count() FROM postgres_database.postgresql_replica_2 on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreplicatedenginearguments-gw7-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.3.2, port:2181, use_ssl:False Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query SELECT count() FROM (SELECT * FROM test_database.postgresql_replica_2) on instance thread 1: insert for 2000-01-02: [8, 10, 2, 3, 2, 5, 5, 1, 8, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/.env --project-name roottestreplicatedenginearguments-gw7 --file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/.env --project-name roottestreplicatedenginearguments-gw7 --file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml up -d --no-recreate] Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SET max_partitions_per_insert_block = 10000000; INSERT INTO test_rename_with_parallel_slow_insert (num,foo2) SELECT number + 0 AS num, number + 1 + 0 AS foo2 FROM numbers_mt(100) on node1 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance thread 0: insert for 2000-01-01: [1, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance thread 1: insert for 2000-01-02: [10, 2, 7, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node5 thread 0: insert for 2000-01-01: [1, 8, 1, 1, 8, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node5 Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Running Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Running Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Running Stderr: Container roottestreplicatedenginearguments-gw7-node1-1 Creating Stderr: Container roottestreplicatedenginearguments-gw7-node1-1 Created Stderr: Container roottestreplicatedenginearguments-gw7-node1-1 Starting Stderr: Container roottestreplicatedenginearguments-gw7-node1-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedenginearguments-gw7-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedenginearguments-gw7-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.3.5... http://localhost:None "GET /v1.46/containers/roottestreplicatedenginearguments-gw7-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/a4099e97d4e8693063cc99b845e63e32ff9b62abb344e5a06376aafc2b8b0af3/json HTTP/1.1" 200 None Executing query SELECT mutation_id, command, parts_to_do, is_done FROM system.mutations WHERE table = 'test_mutations' FORMAT TSVWithNames on node5 mutation_id command parts_to_do is_done mutation_17.txt (UPDATE i = sleepEachRow(2) WHERE 1) 0 1 Executing query SELECT partition, count(name), sum(active), sum(active*rows) FROM system.parts WHERE table ='test_mutations' GROUP BY partition FORMAT TSVWithNames on node5 http://localhost:None "GET /v1.46/containers/a4099e97d4e8693063cc99b845e63e32ff9b62abb344e5a06376aafc2b8b0af3/json HTTP/1.1" 200 None Connecting to 172.16.2.2(172.16.2.2):2281, use_ssl: True partition count(name) sum(active) sum(multiply(active, rows)) 200701 2 1 10 201201 2 1 10 200201 2 1 10 200901 2 1 10 201501 2 1 10 200001 251 4 1182 200501 2 1 10 201001 2 1 10 201301 2 1 10 200601 2 1 10 200301 2 1 10 200101 2 1 10 201401 2 1 10 201101 2 1 10 200401 2 1 10 200801 2 1 10 [gw9] PASSED test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes0] test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes1] Executing query DROP TABLE IF EXISTS test_mutations SYNC on node1 http://localhost:None "GET /v1.46/containers/a4099e97d4e8693063cc99b845e63e32ff9b62abb344e5a06376aafc2b8b0af3/json HTTP/1.1" 200 None Executing query DROP TABLE IF EXISTS test_mutations SYNC on node2 Executing query ALTER TABLE test_rename_with_parallel_slow_insert RENAME COLUMN foo2 to num2 on node1 Executing query DROP TABLE IF EXISTS test_mutations SYNC on node3 http://localhost:None "GET /v1.46/containers/a4099e97d4e8693063cc99b845e63e32ff9b62abb344e5a06376aafc2b8b0af3/json HTTP/1.1" 200 None Executing query DROP TABLE IF EXISTS test_mutations SYNC on node4 http://localhost:None "GET /v1.46/containers/a4099e97d4e8693063cc99b845e63e32ff9b62abb344e5a06376aafc2b8b0af3/json HTTP/1.1" 200 None ClickHouse node1 started Executing query DROP DATABASE IF EXISTS r on node1 Executing query DROP TABLE IF EXISTS test_mutations SYNC on node5 Executing query CREATE DATABASE r ENGINE=Replicated('/clickhouse/databases/r', '{shard}', '{replica}') on node1 Executing query CREATE TABLE r.t1 (x UInt8, y String) ENGINE=ReplicatedMergeTree ORDER BY x on node1 Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node1 Executing query SET database_replicated_allow_replicated_engine_arguments=0; CREATE TABLE r.t2 AS r.t1 on node1 Executing query SHOW CREATE TABLE r.t2 on node1 Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node2 Executing query DROP DATABASE r on node1 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Closing connection to 172.16.2.2:2281 Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.2.3, port:2281, use_ssl:True Connecting to 172.16.2.3(172.16.2.3):2281, use_ssl: True Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node3 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/.env --project-name roottestreplicatedenginearguments-gw7 --file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop --timeout 20] [gw7] PASSED test_replicated_engine_arguments/test.py::test_replicated_engine_with_arguments Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE ReplicatedMergeTree('/clickhouse/{cluster}/tables/test/test_mutations', '{instance}') ORDER BY x PARTITION BY toYYYYMM(d) SETTINGS number_of_free_entries_in_pool_to_execute_mutation=0 on node4 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Closing connection to 172.16.2.3:2281 Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreloadclientcertificate-gw4-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.2.4, port:2281, use_ssl:True Connecting to 172.16.2.4(172.16.2.4):2281, use_ssl: True Executing query CREATE TABLE test_mutations(d Date, x UInt32, i UInt32) ENGINE MergeTree() ORDER BY x PARTITION BY toYYYYMM(d) on node5 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Executing query INSERT INTO test_mutations FORMAT TSV on node3 Received response(xid=1): ['zookeeper', 'clickhouse'] Sending request(xid=2): Close() Closing connection to 172.16.2.4:2281 Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ['zoo1', 'zoo2', 'zoo3'] Executing query SELECT count() FROM system.zookeeper WHERE path = '/' on node1 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Connecting to 172.16.7.2(172.16.7.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['clickhouse', 'keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.7.3, port:2181, use_ssl:False Connecting to 172.16.7.3(172.16.7.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['clickhouse', 'keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query INSERT INTO test_mutations FORMAT TSV on node3 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestremovestalemovingparts-gw1-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.7.4, port:2181, use_ssl:False Connecting to 172.16.7.4(172.16.7.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper', 'clickhouse'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ['zoo1', 'zoo2', 'zoo3'] Executing query SYSTEM START MOVES on ch1 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query DROP TABLE test_remove on ch1 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Command:[docker compose --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --project-name roottestremovestalemovingparts-gw1 --file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/ch1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml stop --timeout 20] [gw1] PASSED test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query ALTER TABLE test_mutations UPDATE i = sleepEachRow(2) WHERE 1 on node3 thread 1: insert for 2000-01-02: [7, 7, 1, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [7, 5, 1, 1, 2, 4, 2, 9, 9, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [8] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT count() FROM test_rename_with_parallel_slow_insert WHERE num2 % 1000 > 0 on node1 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance thread 0: insert for 2000-01-01: [10] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query SELECT count() FROM postgres_database.postgresql_replica_3 on instance Executing query SELECT count() FROM test_rename_with_parallel_slow_insert WHERE num2 % 1000 > 0 on node2 Executing query SELECT count() FROM (SELECT * FROM test_database.postgresql_replica_3) on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Executing query SELECT count() FROM test_rename_with_parallel_slow_insert WHERE num2 % 1000 > 0 on node3 thread 1: insert for 2000-01-02: [9, 2, 9, 10, 10, 6, 4, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query DROP TABLE IF EXISTS test_rename_with_parallel_slow_insert SYNC on node1 thread 0: insert for 2000-01-01: [5, 2, 6, 1, 6, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance Executing query DROP TABLE IF EXISTS test_rename_with_parallel_slow_insert SYNC on node2 thread 1: insert for 2000-01-02: [9, 10, 4, 10, 10, 5, 7, 2, 7, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_slow_insert SYNC on node3 thread 0: insert for 2000-01-01: [10, 9, 9, 3, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Stderr: node2 Skipped - Image is already being pulled by node1 Stderr: node1 Pulling Stderr: node1 Pulled ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/.env --project-name roottestremoteprewhere-gw5 --file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node2/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/.env --project-name roottestremoteprewhere-gw5 --file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node2/docker-compose.yml up -d --no-recreate] Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_slow_insert SYNC on node4 Stderr: Container roottestreplicatedenginearguments-gw7-node1-1 Stopping Stderr: Container roottestreplicatedenginearguments-gw7-node1-1 Stopped Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Stopping Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Stopping Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Stopping Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Stopped Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Stopped Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/.env --project-name roottestreplicatedenginearguments-gw7 --file /ClickHouse/tests/integration/test_replicated_engine_arguments/_instances-0-gw7/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml down --volumes] thread 1: insert for 2000-01-02: [9, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Stderr: node Skipped - Image is already being pulled by zoo2 Stderr: zoo3 Skipped - Image is already being pulled by zoo2 Stderr: zoo1 Skipped - Image is already being pulled by zoo2 Stderr: zoo2 Pulling Stderr: zoo2 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper1/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper1/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper1/coordination', '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper2/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper2/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper2/coordination', '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper3/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper3/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/keeper3/coordination'] Command:[docker compose --project-name roottestreplicatedmergetreeconfig-gw6 --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] thread 0: insert for 2000-01-01: [6, 8, 3, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance [gw3] PASSED test_rename_column/test.py::test_rename_with_parallel_slow_insert test_rename_column/test.py::test_rename_with_parallel_ttl_delete Executing query SELECT value FROM system.build_options WHERE name = 'CXX_FLAGS' on node1 Executing query CREATE TABLE test_rename_with_parallel_ttl_delete ( num UInt32, num2 UInt32 DEFAULT num + 1, time DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_ttl_delete', 'node1') ORDER BY num PARTITION BY num % 100 TTL time + INTERVAL (num2 % 1) SECOND DELETE SETTINGS merge_with_ttl_timeout = 1 on node1 Executing query CREATE TABLE test_rename_with_parallel_ttl_delete ( num UInt32, num2 UInt32 DEFAULT num + 1, time DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_ttl_delete', 'node2') ORDER BY num PARTITION BY num % 100 TTL time + INTERVAL (num2 % 1) SECOND DELETE SETTINGS merge_with_ttl_timeout = 1, merge_with_ttl_timeout = 1 on node2 thread 0: insert for 2000-01-01: [6, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query CREATE TABLE test_rename_with_parallel_ttl_delete ( num UInt32, num2 UInt32 DEFAULT num + 1, time DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_ttl_delete', 'node3') ORDER BY num PARTITION BY num % 100 TTL time + INTERVAL (num2 % 1) SECOND DELETE SETTINGS merge_with_ttl_timeout = 1, merge_with_ttl_timeout = 1, merge_with_ttl_timeout = 1 on node3 thread 1: insert for 2000-01-02: [10, 1, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query CREATE TABLE test_rename_with_parallel_ttl_delete ( num UInt32, num2 UInt32 DEFAULT num + 1, time DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_ttl_delete', 'node4') ORDER BY num PARTITION BY num % 100 TTL time + INTERVAL (num2 % 1) SECOND DELETE SETTINGS merge_with_ttl_timeout = 1, merge_with_ttl_timeout = 1, merge_with_ttl_timeout = 1, merge_with_ttl_timeout = 1 on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Stderr: Container roottestreplicatedenginearguments-gw7-node1-1 Stopping Stderr: Container roottestreplicatedenginearguments-gw7-node1-1 Stopped Stderr: Container roottestreplicatedenginearguments-gw7-node1-1 Removing Stderr: Container roottestreplicatedenginearguments-gw7-node1-1 Removed Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Stopping Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Stopping Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Stopping Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Stopped Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Removing Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Stopped Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Removing Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Stopped Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Removing Stderr: Container roottestreplicatedenginearguments-gw7-zoo2-1 Removed Stderr: Container roottestreplicatedenginearguments-gw7-zoo1-1 Removed Stderr: Container roottestreplicatedenginearguments-gw7-zoo3-1 Removed Stderr: Network roottestreplicatedenginearguments-gw7_default Removing Stderr: Network roottestreplicatedenginearguments-gw7_default Removed Cleanup called Docker networks for project roottestreplicatedenginearguments-gw7 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedenginearguments-gw7 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES thread 1: insert for 2000-01-02: [6, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [9, 2, 8, 10, 8, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Docker volumes for project roottestreplicatedenginearguments-gw7 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedenginearguments-gw7-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicatedenginearguments-gw7 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN time to time2 on node1 Stderr: Network roottestremoteprewhere-gw5_default Creating Stderr: Network roottestremoteprewhere-gw5_default Created Stderr: Container roottestremoteprewhere-gw5-node2-1 Creating Stderr: Container roottestremoteprewhere-gw5-node1-1 Creating Stderr: Container roottestremoteprewhere-gw5-node1-1 Created Stderr: Container roottestremoteprewhere-gw5-node2-1 Created Stderr: Container roottestremoteprewhere-gw5-node1-1 Starting Stderr: Container roottestremoteprewhere-gw5-node2-1 Starting Stderr: Container roottestremoteprewhere-gw5-node2-1 Started Stderr: Container roottestremoteprewhere-gw5-node1-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestremoteprewhere-gw5-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestremoteprewhere-gw5-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.5.3... http://localhost:None "GET /v1.46/containers/roottestremoteprewhere-gw5-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/f33898d7935c9bb9984a7a3ba316cc9bb9d242de38312dada24cb626062de973/json HTTP/1.1" 200 None Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 http://localhost:None "GET /v1.46/containers/f33898d7935c9bb9984a7a3ba316cc9bb9d242de38312dada24cb626062de973/json HTTP/1.1" 200 None Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN time2 to time on node1 http://localhost:None "GET /v1.46/containers/f33898d7935c9bb9984a7a3ba316cc9bb9d242de38312dada24cb626062de973/json HTTP/1.1" 200 None thread 0: insert for 2000-01-01: [2, 3, 2, 5, 2, 9, 9, 2, 9, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 http://localhost:None "GET /v1.46/containers/f33898d7935c9bb9984a7a3ba316cc9bb9d242de38312dada24cb626062de973/json HTTP/1.1" 200 None thread 1: insert for 2000-01-02: [10, 10, 10, 7, 4, 1, 9, 4, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SET max_partitions_per_insert_block = 10000000; SET max_insert_block_size = 256; INSERT INTO test_rename_with_parallel_ttl_delete (num, num2, time) SELECT number AS num, number + 1 AS num2, now() + 10 AS time FROM numbers_mt(10000) on node1 Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance http://localhost:None "GET /v1.46/containers/f33898d7935c9bb9984a7a3ba316cc9bb9d242de38312dada24cb626062de973/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/f33898d7935c9bb9984a7a3ba316cc9bb9d242de38312dada24cb626062de973/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestremoteprewhere-gw5-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestremoteprewhere-gw5-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.5.2... http://localhost:None "GET /v1.46/containers/roottestremoteprewhere-gw5-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/4916b19c66f12c41c68febeef14049778870888237226b39c91cb1749f070d1d/json HTTP/1.1" 200 None ClickHouse node2 started Executing query CREATE TABLE test_table( APIKey UInt32, CustomAttributeId UInt64, ProfileIDHash UInt64, DeviceIDHash UInt64, Data String) ENGINE = SummingMergeTree() ORDER BY (APIKey, CustomAttributeId, ProfileIDHash, DeviceIDHash, intHash32(DeviceIDHash)) on node1 Executing query CREATE TABLE test_table( APIKey UInt32, CustomAttributeId UInt64, ProfileIDHash UInt64, DeviceIDHash UInt64, Data String) ENGINE = SummingMergeTree() ORDER BY (APIKey, CustomAttributeId, ProfileIDHash, DeviceIDHash, intHash32(DeviceIDHash)) on node2 thread 1: insert for 2000-01-02: [4, 7, 6, 3, 3, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [1, 2, 10, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT 1 FROM remote('node{1,2}', default.test_table) WHERE (APIKey = 137715) AND (CustomAttributeId IN (45, 66)) AND (ProfileIDHash != 0) LIMIT 1 on node1 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Stderr:time="2025-04-02T03:33:03Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreplicatedmergetreeconfig-gw6_default Creating Stderr: Network roottestreplicatedmergetreeconfig-gw6_default Created Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Creating Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Creating Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Creating Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Created Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Created Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Created Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Starting Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Starting Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Starting Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Started Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Started Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Started Stderr:time="2025-04-02T03:33:05Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:33:05Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeconfig-gw6-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.3.2, port:2181, use_ssl:False Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Command:[docker compose --env-file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/.env --project-name roottestremoteprewhere-gw5 --file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node2/docker-compose.yml stop --timeout 20] [gw5] PASSED test_remote_prewhere/test.py::test_remote Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 1: insert for 2000-01-02: [1, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 0: insert for 2000-01-01: [6, 3, 10, 3, 3, 7, 10, 6, 6, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance Executing query SELECT count() FROM postgres_database.postgresql_replica_4 on instance Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 1: insert for 2000-01-02: [3, 1, 8, 5, 5, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT count() FROM (SELECT * FROM test_database.postgresql_replica_4) on instance thread 0: insert for 2000-01-01: [3, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT * FROM `test_database`.`postgresql_replica_5` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_5' on instance Executing query select * from `postgres_database`.`postgresql_replica_5` order by key; on instance Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query select * from `test_database`.`postgresql_replica_5` order by key; on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 1: insert for 2000-01-02: [4, 1, 8, 6, 7, 1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [10, 3, 2, 8, 4, 2, 4, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 1: insert for 2000-01-02: [3, 5, 4, 7, 9, 10, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [10, 7, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [9] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 0: insert for 2000-01-01: [4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 1: insert for 2000-01-02: [10, 7, 2, 10, 6, 6, 3, 3, 9, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query select * from `test_database`.`postgresql_replica_5` order by key; on instance Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 0: insert for 2000-01-01: [7, 3, 8, 4, 6, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [9, 4, 5, 5, 6, 3, 2, 7, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [10] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query select * from `test_database`.`postgresql_replica_5` order by key; on instance thread 0: insert for 2000-01-01: [8, 9, 3, 6, 8, 6, 3, 8, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [1, 7, 2, 9, 4, 8, 2, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [4, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [6, 3, 6, 9, 1, 4, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 1: insert for 2000-01-02: [6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 0: insert for 2000-01-01: [9, 5, 8, 6, 4, 1, 1, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Stderr: Container roottestremoteprewhere-gw5-node2-1 Stopping Stderr: Container roottestremoteprewhere-gw5-node1-1 Stopping Stderr: Container roottestremoteprewhere-gw5-node2-1 Stopped Stderr: Container roottestremoteprewhere-gw5-node1-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Executing query select * from `test_database`.`postgresql_replica_5` order by key; on instance Command:[bash -c [ -f /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/.env --project-name roottestremoteprewhere-gw5 --file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node1/docker-compose.yml --file /ClickHouse/tests/integration/test_remote_prewhere/_instances-0-gw5/node2/docker-compose.yml down --volumes] Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query SELECT count() FROM postgres_database.postgresql_replica_5 on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeconfig-gw6-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.3.4, port:2181, use_ssl:False Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query SELECT count() FROM (SELECT * FROM test_database.postgresql_replica_5) on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 thread 1: insert for 2000-01-02: [7, 2, 9, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeconfig-gw6-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.3.3, port:2181, use_ssl:False Connecting to 172.16.3.3(172.16.3.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_concurrent_transactions Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/.env --project-name roottestreplicatedmergetreeconfig-gw6 --file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/.env --project-name roottestreplicatedmergetreeconfig-gw6 --file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml up -d --no-recreate] thread 0: insert for 2000-01-01: [4, 1, 9, 6, 4, 8, 9, 6, 3, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [8, 7, 1, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [1, 5, 9, 3, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Stderr: Container roottestremoteprewhere-gw5-node2-1 Stopping Stderr: Container roottestremoteprewhere-gw5-node1-1 Stopping Stderr: Container roottestremoteprewhere-gw5-node2-1 Stopped Stderr: Container roottestremoteprewhere-gw5-node2-1 Removing Stderr: Container roottestremoteprewhere-gw5-node1-1 Stopped Stderr: Container roottestremoteprewhere-gw5-node1-1 Removing Stderr: Container roottestremoteprewhere-gw5-node2-1 Removed Stderr: Container roottestremoteprewhere-gw5-node1-1 Removed Stderr: Network roottestremoteprewhere-gw5_default Removing Stderr: Network roottestremoteprewhere-gw5_default Removed Cleanup called Docker networks for project roottestremoteprewhere-gw5 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestremoteprewhere-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestremoteprewhere-gw5 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestremoteprewhere-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestremoteprewhere-gw5 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Running Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Running Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Running Stderr: Container roottestreplicatedmergetreeconfig-gw6-node-1 Creating Stderr: Container roottestreplicatedmergetreeconfig-gw6-node-1 Created Stderr: Container roottestreplicatedmergetreeconfig-gw6-node-1 Starting Stderr: Container roottestreplicatedmergetreeconfig-gw6-node-1 Started ClickHouse instance created get_instance_ip instance_name=node http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeconfig-gw6-node-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeconfig-gw6-node-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node, ip: 172.16.3.5... http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeconfig-gw6-node-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/b2bbf06a57b98570086b409cb94f673fa94c9340f4e58203ecefd3f360f7d181/json HTTP/1.1" 200 None Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 test_replicated_database_alter_modify_order_by/test.py::test_alter_modify_order_by Running tests in /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/test.py Cluster start called. is_up=False Docker networks for project roottestreplicateddatabasealtermodifyorderby-gw5 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicateddatabasealtermodifyorderby-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicateddatabasealtermodifyorderby-gw5 are DRIVER VOLUME NAME Cleanup called http://localhost:None "GET /v1.46/containers/b2bbf06a57b98570086b409cb94f673fa94c9340f4e58203ecefd3f360f7d181/json HTTP/1.1" 200 None Docker networks for project roottestreplicateddatabasealtermodifyorderby-gw5 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicateddatabasealtermodifyorderby-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicateddatabasealtermodifyorderby-gw5 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicateddatabasealtermodifyorderby-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] thread 1: insert for 2000-01-02: [4, 7, 4, 6, 8, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Unstopped containers: {} No running containers for project: roottestreplicateddatabasealtermodifyorderby-gw5 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] http://localhost:None "GET /v1.46/containers/b2bbf06a57b98570086b409cb94f673fa94c9340f4e58203ecefd3f360f7d181/json HTTP/1.1" 200 None Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 Setup directory for instance: shard1_node Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/configs/config.xml'] to /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard1_node/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard1_node/database Setup logs dir /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard1_node/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Setup directory for instance: shard2_node Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/configs/config.xml'] to /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard2_node/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard2_node/database Setup logs dir /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard2_node/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/.env --project-name roottestreplicateddatabasealtermodifyorderby-gw5 --file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard1_node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard2_node/docker-compose.yml pull] http://localhost:None "GET /v1.46/containers/b2bbf06a57b98570086b409cb94f673fa94c9340f4e58203ecefd3f360f7d181/json HTTP/1.1" 200 None thread 0: insert for 2000-01-01: [10, 3, 7, 5, 7, 1, 6, 2, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 http://localhost:None "GET /v1.46/containers/b2bbf06a57b98570086b409cb94f673fa94c9340f4e58203ecefd3f360f7d181/json HTTP/1.1" 200 None ClickHouse node started Cluster started Executing query CREATE TABLE test1 (id Int64) ENGINE MergeTree ORDER BY id on node Executing query CREATE TABLE test2 (id Int64) ENGINE ReplicatedMergeTree('/clickhouse/test', 'test') ORDER BY id on node thread 1: insert for 2000-01-02: [6, 9, 2, 10, 7, 3, 4, 1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SHOW CREATE test1 on node Executing query SHOW CREATE test2 on node thread 0: insert for 2000-01-01: [10, 2, 8, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [10, 1, 2, 1, 7, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query DROP TABLE IF EXISTS test1 on node [gw6] PASSED test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query DROP TABLE IF EXISTS test2 on node Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/.env --project-name roottestreplicatedmergetreeconfig-gw6 --file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml stop --timeout 20] Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance thread 0: insert for 2000-01-01: [8] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [5, 7, 9, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_different_data_types Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance thread 0: insert for 2000-01-01: [1, 5, 6, 7, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO postgres_database.test_data_types VALUES (0, -32768, -2147483648, -9223372036854775808, 1.12345, 1.1234567890, 2147483647, 9223372036854775807, '2000-05-12 12:12:12.012345', '2000-05-12', 0.2, 0.2) on instance thread 1: insert for 2000-01-02: [9, 2, 9, 7, 10, 7, 2, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query INSERT INTO postgres_database.test_data_types VALUES (1, -32768, -2147483648, -9223372036854775808, 1.12345, 1.1234567890, 2147483647, 9223372036854775807, '2000-05-12 12:12:12.012345', '2000-05-12', 0.2, 0.2) on instance Executing query INSERT INTO postgres_database.test_data_types VALUES (2, -32768, -2147483648, -9223372036854775808, 1.12345, 1.1234567890, 2147483647, 9223372036854775807, '2000-05-12 12:12:12.012345', '2000-05-12', 0.2, 0.2) on instance Executing query INSERT INTO postgres_database.test_data_types VALUES (3, -32768, -2147483648, -9223372036854775808, 1.12345, 1.1234567890, 2147483647, 9223372036854775807, '2000-05-12 12:12:12.012345', '2000-05-12', 0.2, 0.2) on instance Executing query INSERT INTO postgres_database.test_data_types VALUES (4, -32768, -2147483648, -9223372036854775808, 1.12345, 1.1234567890, 2147483647, 9223372036854775807, '2000-05-12 12:12:12.012345', '2000-05-12', 0.2, 0.2) on instance thread 1: insert for 2000-01-02: [5, 5, 3, 5, 4, 4, 5, 2, 2, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query INSERT INTO postgres_database.test_data_types VALUES (5, -32768, -2147483648, -9223372036854775808, 1.12345, 1.1234567890, 2147483647, 9223372036854775807, '2000-05-12 12:12:12.012345', '2000-05-12', 0.2, 0.2) on instance Executing query INSERT INTO postgres_database.test_data_types VALUES (6, -32768, -2147483648, -9223372036854775808, 1.12345, 1.1234567890, 2147483647, 9223372036854775807, '2000-05-12 12:12:12.012345', '2000-05-12', 0.2, 0.2) on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query INSERT INTO postgres_database.test_data_types VALUES (7, -32768, -2147483648, -9223372036854775808, 1.12345, 1.1234567890, 2147483647, 9223372036854775807, '2000-05-12 12:12:12.012345', '2000-05-12', 0.2, 0.2) on instance Executing query INSERT INTO postgres_database.test_data_types VALUES (8, -32768, -2147483648, -9223372036854775808, 1.12345, 1.1234567890, 2147483647, 9223372036854775807, '2000-05-12 12:12:12.012345', '2000-05-12', 0.2, 0.2) on instance Executing query INSERT INTO postgres_database.test_data_types VALUES (9, -32768, -2147483648, -9223372036854775808, 1.12345, 1.1234567890, 2147483647, 9223372036854775807, '2000-05-12 12:12:12.012345', '2000-05-12', 0.2, 0.2) on instance thread 1: insert for 2000-01-02: [10, 7, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [9, 4, 6, 6, 6, 1, 8, 5, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT * FROM `test_database`.`test_data_types` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_data_types' on instance Executing query select * from `postgres_database`.`test_data_types` order by id; on instance Executing query select * from `test_database`.`test_data_types` order by id; on instance thread 1: insert for 2000-01-02: [1, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [3, 5, 2, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 1: insert for 2000-01-02: [4, 1, 7, 2, 4, 2, 9, 1, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [1, 3, 9, 5, 8, 8, 3, 6, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [6, 8, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query select * from `test_database`.`test_data_types` order by id; on instance Executing query SELECT * FROM test_database.test_data_types ORDER BY id LIMIT 1; on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 0: insert for 2000-01-01: [9, 8, 1, 9, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT * FROM `test_database`.`test_data_types` LIMIT 1 FORMAT Null on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_data_types' on instance Executing query select * from `postgres_database`.`test_data_types` order by id; on instance Executing query select * from `test_database`.`test_data_types` order by id; on instance thread 0: insert for 2000-01-01: [6, 2, 7, 6, 6, 5, 3, 2, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [6, 7, 4, 4, 9, 4, 10, 6, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [4, 6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 1: insert for 2000-01-02: [1, 1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Stderr: Container roottestreplicatedmergetreeconfig-gw6-node-1 Stopping Stderr: Container roottestreplicatedmergetreeconfig-gw6-node-1 Stopped Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Stopping Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Stopping Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Stopping Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Stopped Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Stopped Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/.env --project-name roottestreplicatedmergetreeconfig-gw6 --file /ClickHouse/tests/integration/test_replicated_merge_tree_config/_instances-0-gw6/node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml down --volumes] thread 0: insert for 2000-01-01: [6, 4, 4, 2, 10, 10, 3, 2, 10, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query select * from `test_database`.`test_data_types` order by id; on instance thread 1: insert for 2000-01-02: [1, 4, 1, 10, 6, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Stderr: Container roottestreplicatedmergetreeconfig-gw6-node-1 Stopping Stderr: Container roottestreplicatedmergetreeconfig-gw6-node-1 Stopped Stderr: Container roottestreplicatedmergetreeconfig-gw6-node-1 Removing Stderr: Container roottestreplicatedmergetreeconfig-gw6-node-1 Removed Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Stopping Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Stopping Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Stopping Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Stopped Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Removing Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Stopped Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Removing Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Stopped Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Removing Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo3-1 Removed Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo2-1 Removed Stderr: Container roottestreplicatedmergetreeconfig-gw6-zoo1-1 Removed Stderr: Network roottestreplicatedmergetreeconfig-gw6_default Removing Stderr: Network roottestreplicatedmergetreeconfig-gw6_default Removed Cleanup called Docker networks for project roottestreplicatedmergetreeconfig-gw6 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedmergetreeconfig-gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES thread 0: insert for 2000-01-01: [3, 4, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Docker volumes for project roottestreplicatedmergetreeconfig-gw6 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmergetreeconfig-gw6-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicatedmergetreeconfig-gw6 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:7 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 7 thread 1: insert for 2000-01-02: [3, 9, 6, 4, 7, 9, 5, 8, 5, 2] Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 0: insert for 2000-01-01: [2] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [9, 4, 6, 8, 8, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query select * from `test_database`.`test_data_types` order by id; on instance Executing query INSERT INTO postgres_database.test_array_data_type VALUES (0, ['2000-05-12', '2000-05-12'], ['2000-05-12 12:12:12.012345', '2000-05-12 12:12:12.012345'], [[1.12345], [1.12345], [1.12345]], [[1.1234567891], [1.1234567891], [1.1234567891]], [[[0.11111, 0.11111]], [[0.22222, 0.22222]], [[0.33333, 0.33333]]], [[[1, 1], [1, 1]], [[3, 3], [3, 3]], [[4, 4], [5, 5]]], [[[[['winx', 'winx', 'winx']]]]], [[[1, NULL], [NULL, 1]], [[NULL, NULL], [NULL, NULL]], [[4, 4], [5, 5]]], [[[[NULL]]]], []) on instance Executing query SELECT * FROM `test_database`.`test_array_data_type` LIMIT 1 FORMAT Null on instance thread 0: insert for 2000-01-01: [9, 3, 9, 4, 10, 1, 4, 8, 4, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_array_data_type' on instance Executing query select * from `postgres_database`.`test_array_data_type` order by key; on instance Executing query select * from `test_database`.`test_array_data_type` order by key; on instance Executing query SELECT * FROM test_database.test_array_data_type ORDER BY key; on instance thread 0: insert for 2000-01-01: [2, 8, 7, 6, 3, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_different_data_types Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 1: insert for 2000-01-02: [9, 7, 7, 6, 10, 5, 1, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_drop_database_while_replication_startup_not_finished Executing query INSERT INTO `postgres_database`.postgresql_replica_0 SELECT number, number from numbers(100000) on instance thread 0: insert for 2000-01-01: [6, 7, 4, 4, 2, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [1, 4, 7, 5, 9, 4, 10, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO `postgres_database`.postgresql_replica_1 SELECT number, number from numbers(100000) on instance Executing query INSERT INTO `postgres_database`.postgresql_replica_2 SELECT number, number from numbers(100000) on instance thread 0: insert for 2000-01-01: [4, 8, 10, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [1, 5, 8, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query INSERT INTO `postgres_database`.postgresql_replica_3 SELECT number, number from numbers(100000) on instance thread 0: insert for 2000-01-01: [2, 10, 1, 6, 6, 6, 7, 10, 10, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [7, 2, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO `postgres_database`.postgresql_replica_4 SELECT number, number from numbers(100000) on instance Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance thread 1: insert for 2000-01-02: [1, 7, 1, 7, 6, 1, 10, 6, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [5, 4, 7, 4, 2, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SHOW DATABASES on instance Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance Executing query DROP DATABASE IF EXISTS `test_database` on instance thread 1: insert for 2000-01-02: [3, 2, 8, 4, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 0: insert for 2000-01-01: [1, 6, 4, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SHOW DATABASES on instance thread 1: insert for 2000-01-02: [3, 1, 5, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [6, 9, 1, 7, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN foo2 to foo3 on node2 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN num3 to num2 on node3 Executing query OPTIMIZE TABLE test_rename_with_parallel_ttl_delete on node1 Executing query OPTIMIZE TABLE test_rename_with_parallel_ttl_delete on node2 Executing query OPTIMIZE TABLE test_rename_with_parallel_ttl_delete on node3 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN num3 to num2 on node3 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN foo2 to foo3 on node2 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN num3 to num2 on node3 Executing query OPTIMIZE TABLE test_rename_with_parallel_ttl_delete on node2 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN num3 to num2 on node3 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN num3 to num2 on node3 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance Executing query DROP DATABASE IF EXISTS `test_database` on instance thread 1: insert for 2000-01-02: [5, 2, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [8, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 0: insert for 2000-01-01: [3, 1, 10, 4, 5, 5, 10, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [6, 5, 10, 9, 2, 6, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [3, 1, 10, 6, 6, 3, 1, 4, 9, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Stderr: zoo3 Skipped - Image is already being pulled by shard1_node Stderr: zoo1 Skipped - Image is already being pulled by shard1_node Stderr: zoo2 Skipped - Image is already being pulled by shard1_node Stderr: shard2_node Skipped - Image is already being pulled by shard1_node Stderr: shard1_node Pulling Stderr: shard1_node Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper1/log', '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper1/config', '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper1/coordination', '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper2/log', '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper2/config', '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper2/coordination', '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper3/log', '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper3/config', '/ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/keeper3/coordination'] Command:[docker compose --project-name roottestreplicateddatabasealtermodifyorderby-gw5 --env-file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 1: insert for 2000-01-02: [6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance thread 0: insert for 2000-01-01: [3, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN num2 to foo2 on node1 Executing query OPTIMIZE TABLE test_rename_with_parallel_ttl_delete on node3 thread 0: insert for 2000-01-01: [7, 2, 7, 2, 3, 9, 4, 9, 3, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN num2 to foo2 on node1 Executing query OPTIMIZE TABLE test_rename_with_parallel_ttl_delete on node1 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN num2 to foo2 on node1 Executing query OPTIMIZE TABLE test_rename_with_parallel_ttl_delete on node2 thread 1: insert for 2000-01-02: [6, 8, 7, 9, 1, 3, 4, 8, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN num2 to foo2 on node1 Executing query OPTIMIZE TABLE test_rename_with_parallel_ttl_delete on node1 Executing query OPTIMIZE TABLE test_rename_with_parallel_ttl_delete on node3 thread 0: insert for 2000-01-01: [4, 1, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Stderr: Container roottestremovestalemovingparts-gw1-ch1-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-ch1-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/ch1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/ch1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/.env --project-name roottestremovestalemovingparts-gw1 --file /ClickHouse/tests/integration/test_remove_stale_moving_parts/_instances-0-gw1/ch1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml down --volumes] Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 1: insert for 2000-01-02: [5, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [5, 2, 1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN foo2 to foo3 on node2 Stderr:time="2025-04-02T03:33:21Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreplicateddatabasealtermodifyorderby-gw5_default Creating Stderr: Network roottestreplicateddatabasealtermodifyorderby-gw5_default Created Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Creating Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Creating Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Creating Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Created Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Created Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Created Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Starting Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Starting Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Starting Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Started Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Started Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Started Stderr:time="2025-04-02T03:33:22Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:33:22Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.3.4, port:2181, use_ssl:False Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN foo2 to foo3 on node2 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN foo2 to foo3 on node2 Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN foo2 to num2 on node1 Executing query ALTER TABLE test_rename_with_parallel_ttl_delete RENAME COLUMN foo3 to num2 on node1 Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 0: insert for 2000-01-01: [5, 5, 1, 3, 5, 3, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Stderr: Container roottestremovestalemovingparts-gw1-ch1-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-ch1-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-ch1-1 Removing Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Removing Stderr: Container roottestremovestalemovingparts-gw1-ch1-1 Removed Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Removing Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Removing Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Removing Stderr: Container roottestremovestalemovingparts-gw1-resolver-1 Removed Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Removing Stderr: Container roottestremovestalemovingparts-gw1-zoo2-1 Removed Stderr: Container roottestremovestalemovingparts-gw1-minio1-1 Removed Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Stopping Stderr: Container roottestremovestalemovingparts-gw1-zoo3-1 Removed Stderr: Container roottestremovestalemovingparts-gw1-zoo1-1 Removed Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Removing Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Stopped Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Removing Stderr: Container roottestremovestalemovingparts-gw1-proxy2-1 Removed Stderr: Container roottestremovestalemovingparts-gw1-proxy1-1 Removed Stderr: Volume roottestremovestalemovingparts-gw1_data1-1 Removing Stderr: Network roottestremovestalemovingparts-gw1_default Removing Stderr: Volume roottestremovestalemovingparts-gw1_data1-1 Removed Stderr: Network roottestremovestalemovingparts-gw1_default Removed Cleanup called Docker networks for project roottestremovestalemovingparts-gw1 are NETWORK ID NAME DRIVER SCOPE thread 1: insert for 2000-01-02: [9, 4, 8, 9, 4, 8, 9, 8, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Docker containers for project roottestremovestalemovingparts-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestremovestalemovingparts-gw1 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestremovestalemovingparts-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestremovestalemovingparts-gw1 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:5 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 5 test_replicated_database_cluster_groups/test.py::test_cluster_groups Running tests in /ClickHouse/tests/integration/test_replicated_database_cluster_groups/test.py Cluster start called. is_up=False Docker networks for project roottestreplicateddatabaseclustergroups-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicateddatabaseclustergroups-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicateddatabaseclustergroups-gw1 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestreplicateddatabaseclustergroups-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicateddatabaseclustergroups-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicateddatabaseclustergroups-gw1 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicateddatabaseclustergroups-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicateddatabaseclustergroups-gw1 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] thread 0: insert for 2000-01-01: [2, 1, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance Stdout:5 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 5 Setup directory for instance: main_node_1 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files [] to /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_1/database Setup logs dir /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_1/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Setup directory for instance: main_node_2 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files [] to /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_2/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_2/database Setup logs dir /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_2/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Setup directory for instance: backup_node_1 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_database_cluster_groups/configs/backup_group.xml'] to /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_1/database Setup logs dir /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_1/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Setup directory for instance: backup_node_2 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_database_cluster_groups/configs/backup_group.xml'] to /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_2/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_2/database Setup logs dir /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_2/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Executing query DROP DATABASE IF EXISTS `test_database` on instance Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/.env --project-name roottestreplicateddatabaseclustergroups-gw1 --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_1/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_2/docker-compose.yml pull] thread 1: insert for 2000-01-02: [8, 2, 8, 6, 7, 5, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 0: insert for 2000-01-01: [5, 7, 6, 9, 6, 6, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [6, 6, 9, 6, 6, 7, 1, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT count() FROM test_rename_with_parallel_ttl_delete on node1 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_ttl_delete SYNC on node1 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_ttl_delete SYNC on node2 thread 0: insert for 2000-01-01: [2, 5, 2, 4, 6, 9, 7, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [10, 4, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_ttl_delete SYNC on node3 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_ttl_delete SYNC on node4 thread 0: insert for 2000-01-01: [9, 5, 8, 4, 4, 6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 1: insert for 2000-01-02: [8, 10, 3, 5, 6, 8, 8, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node4 [gw3] PASSED test_rename_column/test.py::test_rename_with_parallel_ttl_delete test_rename_column/test.py::test_rename_with_parallel_ttl_move Executing query SELECT value FROM system.build_options WHERE name = 'CXX_FLAGS' on node1 Executing query CREATE TABLE test_rename_with_parallel_ttl_move ( num UInt32, num2 UInt32 DEFAULT num + 1, time DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_ttl_move', 'node1') ORDER BY num PARTITION BY num % 100 TTL time + INTERVAL (num2 % 1) SECOND TO DISK 'external' SETTINGS storage_policy='default_with_external' on node1 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query CREATE TABLE test_rename_with_parallel_ttl_move ( num UInt32, num2 UInt32 DEFAULT num + 1, time DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_ttl_move', 'node2') ORDER BY num PARTITION BY num % 100 TTL time + INTERVAL (num2 % 1) SECOND TO DISK 'external' SETTINGS storage_policy='default_with_external', storage_policy='default_with_external' on node2 Executing query CREATE TABLE test_rename_with_parallel_ttl_move ( num UInt32, num2 UInt32 DEFAULT num + 1, time DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_ttl_move', 'node3') ORDER BY num PARTITION BY num % 100 TTL time + INTERVAL (num2 % 1) SECOND TO DISK 'external' SETTINGS storage_policy='default_with_external', storage_policy='default_with_external', storage_policy='default_with_external' on node3 Executing query CREATE TABLE test_rename_with_parallel_ttl_move ( num UInt32, num2 UInt32 DEFAULT num + 1, time DateTime ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/test_rename_with_parallel_ttl_move', 'node4') ORDER BY num PARTITION BY num % 100 TTL time + INTERVAL (num2 % 1) SECOND TO DISK 'external' SETTINGS storage_policy='default_with_external', storage_policy='default_with_external', storage_policy='default_with_external', storage_policy='default_with_external' on node4 thread 1: insert for 2000-01-02: [8, 8, 7, 8, 8, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [2, 5, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN time to time2 on node1 Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN time2 to time on node1 Executing query SET max_partitions_per_insert_block = 10000000; SET max_insert_block_size = 256; INSERT INTO test_rename_with_parallel_ttl_move (num, num2, time) SELECT number AS num, number + 1 AS num2, now() + 10 AS time FROM numbers_mt(10000) on node1 thread 1: insert for 2000-01-02: [9, 4, 10, 9, 5, 9, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [9, 8, 9, 9, 3, 4, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance thread 0: insert for 2000-01-01: [10, 6, 5, 1, 8, 8, 10, 7, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SHOW DATABASES on instance thread 1: insert for 2000-01-02: [5, 9, 3, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [10, 3, 8, 1, 8, 2, 4, 10, 4, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [6, 3, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 1: insert for 2000-01-02: [4, 8, 5, 4, 10, 8, 10, 3, 9, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 0: insert for 2000-01-01: [2, 2, 3, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [8, 8, 2, 10, 4, 1, 7, 10, 6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [5, 6, 9, 1, 2, 2, 6, 5, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [4, 1, 10, 7, 3, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 0: insert for 2000-01-01: [4, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.3.3, port:2181, use_ssl:False Connecting to 172.16.3.3(172.16.3.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost thread 1: insert for 2000-01-02: [5, 9, 7, 2, 6, 3, 7, 2, 6, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.3.2, port:2181, use_ssl:False Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() thread 0: insert for 2000-01-01: [10] Connection dropped: socket connection broken Transition to CONNECTING Executing query INSERT INTO test_mutations FORMAT TSV on node3 Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/.env --project-name roottestreplicateddatabasealtermodifyorderby-gw5 --file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard1_node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard2_node/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/.env --project-name roottestreplicateddatabasealtermodifyorderby-gw5 --file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard1_node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard2_node/docker-compose.yml up -d --no-recreate] thread 1: insert for 2000-01-02: [9, 3, 4, 8, 3, 2, 6, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [3, 6, 1, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Running Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Running Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Running Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1 Creating Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1 Creating Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1 Created Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1 Created Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1 Starting Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1 Starting Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1 Started Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1 Started ClickHouse instance created get_instance_ip instance_name=shard1_node Executing query DROP DATABASE IF EXISTS "postgres_database" on instance http://localhost:None "GET /v1.46/containers/roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1/json HTTP/1.1" 200 None [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_drop_database_while_replication_startup_not_finished get_instance_ip instance_name=shard1_node http://localhost:None "GET /v1.46/containers/roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in shard1_node, ip: 172.16.3.6... http://localhost:None "GET /v1.46/containers/roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/0cb3e66fcbe6e1e67d0a69ebd32fe4c4c70561bd558528461017af883321a181/json HTTP/1.1" 200 None Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 1: insert for 2000-01-02: [2, 9, 4, 1, 5, 5, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [2, 7, 6, 10, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 http://localhost:None "GET /v1.46/containers/0cb3e66fcbe6e1e67d0a69ebd32fe4c4c70561bd558528461017af883321a181/json HTTP/1.1" 200 None Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance http://localhost:None "GET /v1.46/containers/0cb3e66fcbe6e1e67d0a69ebd32fe4c4c70561bd558528461017af883321a181/json HTTP/1.1" 200 None test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_all_database_tables Executing query INSERT INTO `postgres_database`.postgresql_replica_0 SELECT number, number from numbers(50) on instance http://localhost:None "GET /v1.46/containers/0cb3e66fcbe6e1e67d0a69ebd32fe4c4c70561bd558528461017af883321a181/json HTTP/1.1" 200 None Executing query INSERT INTO `postgres_database`.postgresql_replica_1 SELECT number, number from numbers(50) on instance http://localhost:None "GET /v1.46/containers/0cb3e66fcbe6e1e67d0a69ebd32fe4c4c70561bd558528461017af883321a181/json HTTP/1.1" 200 None ClickHouse shard1_node started get_instance_ip instance_name=shard2_node http://localhost:None "GET /v1.46/containers/roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1/json HTTP/1.1" 200 None get_instance_ip instance_name=shard2_node http://localhost:None "GET /v1.46/containers/roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in shard2_node, ip: 172.16.3.5... http://localhost:None "GET /v1.46/containers/roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/1d06bc366bb4dcb49d01ead71410c636dcd5978203d2ead7c8192b381697e652/json HTTP/1.1" 200 None ClickHouse shard2_node started Executing query DROP DATABASE IF EXISTS alter_modify_order_by SYNC; on shard1_node thread 1: insert for 2000-01-02: [4, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [9, 6, 1, 9, 8, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query DROP DATABASE IF EXISTS alter_modify_order_by SYNC; on shard2_node Executing query INSERT INTO `postgres_database`.postgresql_replica_2 SELECT number, number from numbers(50) on instance Executing query CREATE DATABASE alter_modify_order_by ENGINE = Replicated('/test/database/alter_modify_order_by', '{shard}', '{replica}'); on shard1_node Executing query INSERT INTO `postgres_database`.postgresql_replica_3 SELECT number, number from numbers(50) on instance Executing query CREATE TABLE alter_modify_order_by.t1 (id Int64, score Int64) ENGINE = ReplicatedMergeTree('/test/tables/{uuid}/{shard}', '{replica}') ORDER BY (id); on shard1_node Executing query INSERT INTO `postgres_database`.postgresql_replica_4 SELECT number, number from numbers(50) on instance Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query ALTER TABLE alter_modify_order_by.t1 modify order by (id); on shard1_node Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance thread 1: insert for 2000-01-02: [3, 3, 8, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SHOW DATABASES on instance thread 0: insert for 2000-01-01: [6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query CREATE DATABASE alter_modify_order_by ENGINE = Replicated('/test/database/alter_modify_order_by', '{shard}', '{replica}'); on shard2_node Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query select count() from system.tables where database = 'alter_modify_order_by'; on shard1_node Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query select count() from system.tables where database = 'alter_modify_order_by'; on shard2_node Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance thread 1: insert for 2000-01-02: [10, 9, 8, 5, 3, 9, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query show create table alter_modify_order_by.t1; on shard1_node Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query show create table alter_modify_order_by.t1; on shard2_node Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Executing query DROP DATABASE IF EXISTS alter_modify_order_by SYNC; on shard1_node thread 0: insert for 2000-01-01: [2, 2, 3, 6, 5, 3, 3, 8, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Executing query DROP DATABASE IF EXISTS alter_modify_order_by SYNC; on shard2_node Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/.env --project-name roottestreplicateddatabasealtermodifyorderby-gw5 --file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard1_node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard2_node/docker-compose.yml stop --timeout 20] [gw5] PASSED test_replicated_database_alter_modify_order_by/test.py::test_alter_modify_order_by thread 1: insert for 2000-01-02: [3, 2, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN time to time2 on node1 Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance thread 0: insert for 2000-01-01: [4, 1, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 1: insert for 2000-01-02: [6, 5, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance thread 0: insert for 2000-01-01: [5, 6, 5, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance thread 1: insert for 2000-01-02: [1, 2, 9, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [10, 6, 6, 5, 2, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance Executing query SELECT count() FROM system.tables WHERE database = 'test_database'; on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1 Stopping Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1 Stopping Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1 Stopped Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1 Stopped Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Stopping Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Stopping Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Stopping Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Stopped Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Stopped Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard1_node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard1_node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_all_database_tables Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard2_node/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard2_node/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/.env --project-name roottestreplicateddatabasealtermodifyorderby-gw5 --file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard1_node/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_database_alter_modify_order_by/_instances-0-gw5/shard2_node/docker-compose.yml down --volumes] thread 0: insert for 2000-01-01: [10, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [6, 9, 2, 6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_subset_of_database_tables Executing query INSERT INTO `postgres_database`.postgresql_replica_0 SELECT number, number from numbers(50) on instance thread 1: insert for 2000-01-02: [8, 8, 6, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO `postgres_database`.postgresql_replica_1 SELECT number, number from numbers(50) on instance thread 0: insert for 2000-01-01: [9, 1, 5, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO `postgres_database`.postgresql_replica_2 SELECT number, number from numbers(50) on instance Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1 Stopping Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1 Stopping Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1 Stopped Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1 Removing Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1 Stopped Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1 Removing Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard2_node-1 Removed Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-shard1_node-1 Removed Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Stopping Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Stopping Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Stopping Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Stopped Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Removing Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Stopped Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Removing Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Stopped Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Removing Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo1-1 Removed Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo2-1 Removed Stderr: Container roottestreplicateddatabasealtermodifyorderby-gw5-zoo3-1 Removed Stderr: Network roottestreplicateddatabasealtermodifyorderby-gw5_default Removing Stderr: Network roottestreplicateddatabasealtermodifyorderby-gw5_default Removed Cleanup called Executing query INSERT INTO `postgres_database`.postgresql_replica_3 SELECT number, number from numbers(50) on instance Docker networks for project roottestreplicateddatabasealtermodifyorderby-gw5 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicateddatabasealtermodifyorderby-gw5 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicateddatabasealtermodifyorderby-gw5 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicateddatabasealtermodifyorderby-gw5-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicateddatabasealtermodifyorderby-gw5 Trying to prune unused networks... Executing query INSERT INTO `postgres_database`.postgresql_replica_4 SELECT number, number from numbers(50) on instance Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:5 Command:[docker volume prune -f] thread 1: insert for 2000-01-02: [4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query INSERT INTO `postgres_database`.postgresql_replica_5 SELECT number, number from numbers(50) on instance Stdout:Total reclaimed space: 0B Volumes pruned: 5 thread 0: insert for 2000-01-01: [7, 7, 5, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query INSERT INTO `postgres_database`.postgresql_replica_6 SELECT number, number from numbers(50) on instance Executing query INSERT INTO `postgres_database`.postgresql_replica_7 SELECT number, number from numbers(50) on instance Executing query INSERT INTO `postgres_database`.postgresql_replica_8 SELECT number, number from numbers(50) on instance thread 1: insert for 2000-01-02: [8, 2, 1, 2, 7, 1, 6, 9, 9, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query INSERT INTO `postgres_database`.postgresql_replica_9 SELECT number, number from numbers(50) on instance thread 0: insert for 2000-01-01: [1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') SETTINGS materialized_postgresql_tables_list = 'postgresql_replica_0, postgresql_replica_1, postgresql_replica_2, postgresql_replica_3, postgresql_replica_4' on instance Executing query SHOW DATABASES on instance thread 0: insert for 2000-01-01: [7, 10, 7, 7, 1, 2, 2, 8, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [9, 2, 8, 5, 6, 4, 4, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 0: insert for 2000-01-01: [4, 10, 2, 3, 3, 7, 1, 4, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [10, 1, 1, 3, 2, 4, 4, 2, 8, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Stderr: zoo1 Skipped - Image is already being pulled by zoo3 Stderr: main_node_2 Skipped - Image is already being pulled by zoo3 Stderr: backup_node_1 Skipped - Image is already being pulled by zoo3 Stderr: backup_node_2 Skipped - Image is already being pulled by zoo3 Stderr: main_node_1 Skipped - Image is already being pulled by zoo3 Stderr: zoo2 Skipped - Image is already being pulled by zoo3 Stderr: zoo3 Pulling Stderr: zoo3 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper1/log', '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper1/config', '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper1/coordination', '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper2/log', '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper2/config', '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper2/coordination', '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper3/log', '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper3/config', '/ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/keeper3/coordination'] Command:[docker compose --project-name roottestreplicateddatabaseclustergroups-gw1 --env-file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] thread 0: insert for 2000-01-01: [7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance thread 1: insert for 2000-01-02: [4, 4, 10, 4, 3, 4, 5, 8, 4, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance thread 0: insert for 2000-01-01: [3, 7, 3, 2, 2, 8, 1, 6, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 1: insert for 2000-01-02: [5, 8, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance Stderr:time="2025-04-02T03:33:34Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreplicateddatabaseclustergroups-gw1_default Creating Stderr: Network roottestreplicateddatabaseclustergroups-gw1_default Created Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Creating Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Creating Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Creating Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Created Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Created Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Created Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Starting Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Starting Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Starting Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Started Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Started Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Started Stderr:time="2025-04-02T03:33:34Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:33:34Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.3.2, port:2181, use_ssl:False Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Connection dropped: socket connection error: Connection refused Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance thread 0: insert for 2000-01-01: [7, 3, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused thread 1: insert for 2000-01-02: [9, 1, 10, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN num2 to foo2 on node1 Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN foo2 to foo3 on node2 Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN num3 to num2 on node3 Executing query SELECT count() FROM system.tables WHERE database = 'test_database'; on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN foo2 to foo3 on node2 Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN num3 to num2 on node3 Executing query SHOW TABLES FROM test_database on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN num3 to num2 on node3 Executing query INSERT INTO postgres_database.postgresql_replica_0 SELECT 50 + number, 0 from numbers(100) on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN num3 to num2 on node3 Executing query INSERT INTO postgres_database.postgresql_replica_1 SELECT 50 + number, 1 from numbers(100) on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN num3 to num2 on node3 Executing query INSERT INTO postgres_database.postgresql_replica_2 SELECT 50 + number, 2 from numbers(100) on instance thread 0: insert for 2000-01-01: [10, 7, 9, 7, 8, 7, 8, 4, 9, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Executing query INSERT INTO postgres_database.postgresql_replica_3 SELECT 50 + number, 3 from numbers(100) on instance Connection dropped: socket connection error: Connection refused Executing query INSERT INTO postgres_database.postgresql_replica_4 SELECT 50 + number, 4 from numbers(100) on instance thread 1: insert for 2000-01-02: [6, 8, 1, 9, 8, 6, 2, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query INSERT INTO postgres_database.postgresql_replica_5 SELECT 50 + number, 5 from numbers(100) on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query INSERT INTO postgres_database.postgresql_replica_6 SELECT 50 + number, 6 from numbers(100) on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query INSERT INTO postgres_database.postgresql_replica_7 SELECT 50 + number, 7 from numbers(100) on instance Executing query INSERT INTO postgres_database.postgresql_replica_8 SELECT 50 + number, 8 from numbers(100) on instance thread 1: insert for 2000-01-02: [3, 1, 6] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [2, 1, 6, 6, 5, 1, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO postgres_database.postgresql_replica_9 SELECT 50 + number, 9 from numbers(100) on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN num2 to foo2 on node1 Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN num2 to foo2 on node1 Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN num2 to foo2 on node1 Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN num2 to foo2 on node1 Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance thread 0: insert for 2000-01-01: [3, 8, 6, 5, 10, 9, 6, 1, 3, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [6, 1, 8, 1, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance thread 1: insert for 2000-01-02: [7, 4, 10, 3, 4, 5, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance thread 0: insert for 2000-01-01: [5, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN foo2 to foo3 on node2 Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN foo2 to foo3 on node2 Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN foo2 to foo3 on node2 thread 1: insert for 2000-01-02: [5, 9, 5, 2, 6, 4, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query SELECT disk_name FROM system.parts WHERE table == 'test_rename_with_parallel_ttl_move' AND active=1 ORDER BY modification_time on node1 Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN foo2 to num2 on node1 Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance thread 0: insert for 2000-01-01: [8, 5, 3, 7, 8, 7, 6, 9, 10, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query ALTER TABLE test_rename_with_parallel_ttl_move RENAME COLUMN foo3 to num2 on node1 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance thread 1: insert for 2000-01-02: [6, 9, 8, 10, 3, 6, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_subset_of_database_tables Executing query DROP DATABASE IF EXISTS "postgres_database" on instance thread 0: insert for 2000-01-01: [9, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 test_postgresql_replica_database_engine_1/test.py::test_many_concurrent_queries Executing query INSERT INTO `postgres_database`.test_many_conc_0 SELECT number, number from numbers(10000) on instance thread 1: insert for 2000-01-02: [2, 2, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query INSERT INTO `postgres_database`.test_many_conc_1 SELECT number, number from numbers(10000) on instance thread 0: insert for 2000-01-01: [8, 10, 4, 6, 5, 4, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO `postgres_database`.test_many_conc_2 SELECT number, number from numbers(10000) on instance Connecting to 172.16.3.2(172.16.3.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query INSERT INTO `postgres_database`.test_many_conc_3 SELECT number, number from numbers(10000) on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.3.4, port:2181, use_ssl:False Connecting to 172.16.3.4(172.16.3.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query INSERT INTO `postgres_database`.test_many_conc_4 SELECT number, number from numbers(10000) on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.3.3, port:2181, use_ssl:False Connecting to 172.16.3.3(172.16.3.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query SELECT count() FROM test_rename_with_parallel_ttl_move WHERE num2 % 1000 > 0 on node1 thread 1: insert for 2000-01-02: [10, 1, 6, 1, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/.env --project-name roottestreplicateddatabaseclustergroups-gw1 --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_1/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_2/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/.env --project-name roottestreplicateddatabaseclustergroups-gw1 --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_1/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_2/docker-compose.yml up -d --no-recreate] Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Executing query DROP TABLE IF EXISTS test_rename_with_parallel_ttl_move SYNC on node1 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_ttl_move SYNC on node2 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 0: insert for 2000-01-01: [4, 3, 7, 2, 7, 5, 1, 2, 9] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_ttl_move SYNC on node3 thread 1: insert for 2000-01-02: [7, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query DROP TABLE IF EXISTS test_rename_with_parallel_ttl_move SYNC on node4 [gw3] PASSED test_rename_column/test.py::test_rename_with_parallel_ttl_move Command:[docker compose --env-file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/.env --project-name roottestrenamecolumn-gw3 --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node4/docker-compose.yml stop --timeout 20] Executing query INSERT INTO postgres_database.test_many_conc_0 SELECT 0*10000*(10000 + number), number from numbers(1000) on instance thread 0: insert for 2000-01-01: [4, 5, 5, 10, 6, 8, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 1: insert for 2000-01-02: [9, 6, 7, 5, 6, 6, 4, 2] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Running Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Running Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Running Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 Creating Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 Creating Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_1-1 Creating Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 Creating Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_1-1 Created Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 Created Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 Created Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 Created Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 Starting Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_1-1 Starting Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 Starting Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 Starting Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 Started Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_1-1 Started Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 Started Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 Started ClickHouse instance created get_instance_ip instance_name=main_node_1 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-main_node_1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=main_node_1 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-main_node_1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in main_node_1, ip: 172.16.3.5... http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-main_node_1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/a2d041d0931589d595016a6a11c7270ad6db36f4de7e975310ee60eb4bb9e616/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/a2d041d0931589d595016a6a11c7270ad6db36f4de7e975310ee60eb4bb9e616/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/a2d041d0931589d595016a6a11c7270ad6db36f4de7e975310ee60eb4bb9e616/json HTTP/1.1" 200 None thread 0: insert for 2000-01-01: [5, 5, 6, 6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 http://localhost:None "GET /v1.46/containers/a2d041d0931589d595016a6a11c7270ad6db36f4de7e975310ee60eb4bb9e616/json HTTP/1.1" 200 None ClickHouse main_node_1 started get_instance_ip instance_name=main_node_2 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-main_node_2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=main_node_2 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-main_node_2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in main_node_2, ip: 172.16.3.8... http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-main_node_2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/e6e7f63f064b4e648ee2a6231afe00cd637ab788a5e128547ac0f983d3f6197b/json HTTP/1.1" 200 None Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 thread 1: insert for 2000-01-02: [9, 5, 5, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 http://localhost:None "GET /v1.46/containers/e6e7f63f064b4e648ee2a6231afe00cd637ab788a5e128547ac0f983d3f6197b/json HTTP/1.1" 200 None ClickHouse main_node_2 started get_instance_ip instance_name=backup_node_1 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=backup_node_1 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in backup_node_1, ip: 172.16.3.7... http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/607975d4cee32123fcc1c2906056862d793d854e12a8a53a65ec42d0c6007271/json HTTP/1.1" 200 None ClickHouse backup_node_1 started get_instance_ip instance_name=backup_node_2 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=backup_node_2 http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in backup_node_2, ip: 172.16.3.6... http://localhost:None "GET /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/f26787b88db72311056a784f51956e2bab3898cb51a581c82e72bd937b5b255f/json HTTP/1.1" 200 None ClickHouse backup_node_2 started Executing query CREATE DATABASE cluster_groups ENGINE = Replicated('/test/cluster_groups', '1', '1'); on main_node_1 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query CREATE DATABASE cluster_groups ENGINE = Replicated('/test/cluster_groups', '1', '2'); on main_node_2 Executing query CREATE DATABASE cluster_groups ENGINE = Replicated('/test/cluster_groups', '1', '3'); on backup_node_1 Executing query CREATE DATABASE cluster_groups ENGINE = Replicated('/test/cluster_groups', '1', '4'); on backup_node_2 thread 0: insert for 2000-01-01: [8, 6, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query SELECT host_name from system.clusters WHERE cluster = 'cluster_groups' ORDER BY host_name on main_node_1 Executing query INSERT INTO postgres_database.test_many_conc_1 SELECT 1*10000*(10000 + number), number from numbers(1000) on instance thread 1: insert for 2000-01-02: [3] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT host_name from system.clusters WHERE cluster = 'cluster_groups' ORDER BY host_name on main_node_2 Executing query INSERT INTO postgres_database.test_many_conc_1 SELECT 1*10000*(10001 + number), number from numbers(1000) on instance Executing query SELECT host_name from system.clusters WHERE cluster = 'cluster_groups' ORDER BY host_name on backup_node_1 Executing query SELECT host_name from system.clusters WHERE cluster = 'cluster_groups' ORDER BY host_name on backup_node_2 thread 0: insert for 2000-01-01: [8, 7, 10, 5, 1, 8, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 bash -c ps -C clickhouse] Stdout: PID TTY TIME CMD Stdout: 9 ? 00:00:00 clickhouse run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill clickhouse'] Command:[docker exec -u root roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 bash -c pkill clickhouse] run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:9 thread 1: insert for 2000-01-02: [1, 3, 3, 6, 7, 4, 8, 8, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 thread 0: insert for 2000-01-01: [3, 5, 10, 10, 9, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 thread 1: insert for 2000-01-02: [2, 1, 2, 7, 5, 10, 1, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [2, 9, 6, 6, 6, 2, 6, 4] Executing query INSERT INTO test_mutations FORMAT TSV on node4 Executing query INSERT INTO postgres_database.test_many_conc_2 SELECT 2*10000*(10000 + number), number from numbers(1000) on instance Executing query INSERT INTO postgres_database.test_many_conc_3 SELECT 3*10000*(10000 + number), number from numbers(1000) on instance Executing query INSERT INTO postgres_database.test_many_conc_2 SELECT 2*10000*(10001 + number), number from numbers(1000) on instance thread 1: insert for 2000-01-02: [3, 3, 8, 10, 8, 7, 9, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO postgres_database.test_many_conc_3 SELECT 3*10000*(10001 + number), number from numbers(1000) on instance thread 0: insert for 2000-01-01: [4, 4, 1, 5, 4, 8, 4, 5, 5, 5] Executing query INSERT INTO test_mutations FORMAT TSV on node4 run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:9 thread 1: insert for 2000-01-02: [7, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Executing query INSERT INTO postgres_database.test_many_conc_4 SELECT 4*10000*(10000 + number), number from numbers(1000) on instance Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query INSERT INTO postgres_database.test_many_conc_4 SELECT 4*10000*(10001 + number), number from numbers(1000) on instance thread 0: insert for 2000-01-01: [6, 7] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [6, 9, 7, 5, 5, 3, 5, 1, 5, 1] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 0: insert for 2000-01-01: [8, 8, 8, 3, 9, 3, 8, 7, 8] Executing query INSERT INTO test_mutations FORMAT TSV on node3 thread 1: insert for 2000-01-02: [2, 5, 1, 3, 8, 7, 10, 2, 10] Executing query INSERT INTO test_mutations FORMAT TSV on node4 run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] thread 0: insert for 2000-01-01: [10, 8, 3, 3, 7, 2, 2, 3] Executing query INSERT INTO test_mutations FORMAT TSV on node3 Stdout:9 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node3 Executing query SELECT sum(is_done) FROM system.mutations WHERE table = 'test_mutations' on node4 Executing query SELECT mutation_id, command, parts_to_do, is_done FROM system.mutations WHERE table = 'test_mutations' FORMAT TSVWithNames on node3 mutation_id command parts_to_do is_done 0000000000 (UPDATE i = sleepEachRow(2) WHERE 1) 0 1 Executing query SELECT partition, count(name), sum(active), sum(active*rows) FROM system.parts WHERE table ='test_mutations' GROUP BY partition FORMAT TSVWithNames on node3 partition count(name) sum(active) sum(multiply(active, rows)) 200701 2 1 10 201201 2 1 10 200201 2 1 10 200901 2 1 10 201501 2 1 10 200001 244 2 1142 200501 2 1 10 201001 2 1 10 201301 2 1 10 200601 2 1 10 200301 2 1 10 200101 2 1 10 201401 2 1 10 201101 2 1 10 200401 2 1 10 200801 2 1 10 Executing query SELECT mutation_id, command, parts_to_do, is_done FROM system.mutations WHERE table = 'test_mutations' FORMAT TSVWithNames on node4 mutation_id command parts_to_do is_done 0000000000 (UPDATE i = sleepEachRow(2) WHERE 1) 0 1 Executing query SELECT partition, count(name), sum(active), sum(active*rows) FROM system.parts WHERE table ='test_mutations' GROUP BY partition FORMAT TSVWithNames on node4 partition count(name) sum(active) sum(multiply(active, rows)) 200701 2 1 10 201201 2 1 10 200201 2 1 10 200901 2 1 10 201501 2 1 10 200001 244 2 1142 200501 2 1 10 201001 2 1 10 201301 2 1 10 200601 2 1 10 200301 2 1 10 200101 2 1 10 201401 2 1 10 201101 2 1 10 200401 2 1 10 200801 2 1 10 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/.env --project-name roottestreplicatedmutations-gw9 --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node5/docker-compose.yml stop --timeout 20] [gw9] PASSED test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes1] run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:9 run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps -C clickhouse] Stdout: PID TTY TIME CMD Stdout: 10 ? 00:00:00 clickhouse run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill clickhouse'] Command:[docker exec -u root roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c pkill clickhouse] run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:10 run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:10 Stderr: Container roottestrenamecolumn-gw3-node3-1 Stopping Stderr: Container roottestrenamecolumn-gw3-node2-1 Stopping Stderr: Container roottestrenamecolumn-gw3-node4-1 Stopping Stderr: Container roottestrenamecolumn-gw3-node1-1 Stopping Stderr: Container roottestrenamecolumn-gw3-node4-1 Stopped Stderr: Container roottestrenamecolumn-gw3-node3-1 Stopped Stderr: Container roottestrenamecolumn-gw3-node1-1 Stopped Stderr: Container roottestrenamecolumn-gw3-node2-1 Stopped Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Stopping Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Stopping Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Stopping Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Stopped Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Stopped Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node4/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node4/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/.env --project-name roottestrenamecolumn-gw3 --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_rename_column/_instances-0-gw3/node4/docker-compose.yml down --volumes] Executing query INSERT INTO postgres_database.test_many_conc_0 SELECT 50001 + number, number from numbers(5000) on instance Executing query INSERT INTO postgres_database.test_many_conc_1 SELECT 50002 + number, number from numbers(5000) on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stderr: Container roottestrenamecolumn-gw3-node4-1 Stopping Stderr: Container roottestrenamecolumn-gw3-node3-1 Stopping Stderr: Container roottestrenamecolumn-gw3-node1-1 Stopping Stderr: Container roottestrenamecolumn-gw3-node2-1 Stopping Stdout:10 Stderr: Container roottestrenamecolumn-gw3-node4-1 Stopped Stderr: Container roottestrenamecolumn-gw3-node4-1 Removing Stderr: Container roottestrenamecolumn-gw3-node1-1 Stopped Stderr: Container roottestrenamecolumn-gw3-node1-1 Removing Stderr: Container roottestrenamecolumn-gw3-node3-1 Stopped Stderr: Container roottestrenamecolumn-gw3-node3-1 Removing Stderr: Container roottestrenamecolumn-gw3-node2-1 Stopped Stderr: Container roottestrenamecolumn-gw3-node2-1 Removing Stderr: Container roottestrenamecolumn-gw3-node1-1 Removed Stderr: Container roottestrenamecolumn-gw3-node4-1 Removed Stderr: Container roottestrenamecolumn-gw3-node2-1 Removed Stderr: Container roottestrenamecolumn-gw3-node3-1 Removed Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Stopping Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Stopping Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Stopping Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Stopped Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Removing Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Stopped Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Removing Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Stopped Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Removing Stderr: Container roottestrenamecolumn-gw3-zoo3-1 Removed Stderr: Container roottestrenamecolumn-gw3-zoo2-1 Removed Stderr: Container roottestrenamecolumn-gw3-zoo1-1 Removed Stderr: Network roottestrenamecolumn-gw3_default Removing Stderr: Network roottestrenamecolumn-gw3_default Removed Cleanup called Docker networks for project roottestrenamecolumn-gw3 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestrenamecolumn-gw3 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestrenamecolumn-gw3 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestrenamecolumn-gw3-.*-1$' --format '{{.ID}}:{{.Names}}'] Executing query INSERT INTO postgres_database.test_many_conc_2 SELECT 50003 + number, number from numbers(5000) on instance Unstopped containers: {} No running containers for project: roottestrenamecolumn-gw3 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:5 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 5 Executing query INSERT INTO postgres_database.test_many_conc_3 SELECT 50004 + number, number from numbers(5000) on instance Executing query INSERT INTO postgres_database.test_many_conc_4 SELECT 50005 + number, number from numbers(5000) on instance Executing query SELECT * FROM `test_database`.`test_many_conc_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_many_conc_0' on instance Executing query select * from `postgres_database`.`test_many_conc_0` order by key; on instance Executing query select * from `test_database`.`test_many_conc_0` order by key; on instance Executing query SELECT count() FROM postgres_database.test_many_conc_0 on instance Executing query SELECT count() FROM (SELECT * FROM test_database.test_many_conc_0) on instance Executing query SELECT * FROM `test_database`.`test_many_conc_1` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_many_conc_1' on instance Executing query select * from `postgres_database`.`test_many_conc_1` order by key; on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query select * from `test_database`.`test_many_conc_1` order by key; on instance Executing query CREATE TABLE cluster_groups.table_1 (d Date, k UInt64) ENGINE=ReplicatedMergeTree ORDER BY k PARTITION BY toYYYYMM(d); on main_node_1 Executing query SELECT count() FROM postgres_database.test_many_conc_1 on instance Executing query SELECT count() FROM (SELECT * FROM test_database.test_many_conc_1) on instance Executing query SELECT * FROM `test_database`.`test_many_conc_2` LIMIT 1 FORMAT Null on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps -C clickhouse] Stdout: PID TTY TIME CMD Stdout: 10 ? 00:00:00 clickhouse run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill clickhouse'] Command:[docker exec -u root roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c pkill clickhouse] Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_many_conc_2' on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:10 Executing query select * from `postgres_database`.`test_many_conc_2` order by key; on instance Executing query select * from `test_database`.`test_many_conc_2` order by key; on instance Executing query SELECT count() FROM postgres_database.test_many_conc_2 on instance Executing query SELECT count() FROM (SELECT * FROM test_database.test_many_conc_2) on instance Executing query SELECT * FROM `test_database`.`test_many_conc_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_many_conc_3' on instance Executing query select * from `postgres_database`.`test_many_conc_3` order by key; on instance Executing query select * from `test_database`.`test_many_conc_3` order by key; on instance Executing query SELECT count() FROM postgres_database.test_many_conc_3 on instance Executing query SELECT count() FROM (SELECT * FROM test_database.test_many_conc_3) on instance Executing query SELECT * FROM `test_database`.`test_many_conc_4` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_many_conc_4' on instance Executing query select * from `postgres_database`.`test_many_conc_4` order by key; on instance Executing query select * from `test_database`.`test_many_conc_4` order by key; on instance Executing query SELECT count() FROM postgres_database.test_many_conc_4 on instance Executing query SELECT count() FROM (SELECT * FROM test_database.test_many_conc_4) on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:10 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_many_concurrent_queries Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_multiple_databases Executing query DROP DATABASE IF EXISTS "postgres_database_1" on instance Executing query CREATE DATABASE "postgres_database_1" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database_1', 'postgres', 'mysecretpassword') on instance Executing query DROP DATABASE IF EXISTS "postgres_database_2" on instance Executing query CREATE DATABASE "postgres_database_2" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database_2', 'postgres', 'mysecretpassword') on instance Executing query INSERT INTO postgres_database_1.postgresql_replica_0 SELECT number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_1.postgresql_replica_1 SELECT number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_1.postgresql_replica_2 SELECT number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_1.postgresql_replica_3 SELECT number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_1.postgresql_replica_4 SELECT number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_2.postgresql_replica_0 SELECT number, number from numbers(50) on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:10 Executing query INSERT INTO postgres_database_2.postgresql_replica_1 SELECT number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_2.postgresql_replica_2 SELECT number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_2.postgresql_replica_3 SELECT number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_2.postgresql_replica_4 SELECT number, number from numbers(50) on instance Executing query SELECT name FROM system.tables WHERE database = 'postgres_database_1'; on instance Executing query SELECT name FROM system.tables WHERE database = 'postgres_database_2'; on instance Executing query DROP DATABASE IF EXISTS `test_database_1` on instance Executing query CREATE DATABASE `test_database_1` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database_1', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Executing query DROP DATABASE IF EXISTS `test_database_2` on instance Executing query CREATE DATABASE `test_database_2` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database_2', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Executing query INSERT INTO postgres_database_1.postgresql_replica_0 SELECT 50 + number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_1.postgresql_replica_1 SELECT 50 + number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_1.postgresql_replica_2 SELECT 50 + number, number from numbers(50) on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query INSERT INTO postgres_database_1.postgresql_replica_3 SELECT 50 + number, number from numbers(50) on instance Executing query CREATE TABLE cluster_groups.table_2 (d Date, k UInt64) ENGINE=ReplicatedMergeTree ORDER BY k PARTITION BY toYYYYMM(d); on main_node_1 Executing query INSERT INTO postgres_database_1.postgresql_replica_4 SELECT 50 + number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_2.postgresql_replica_0 SELECT 50 + number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_2.postgresql_replica_1 SELECT 50 + number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_2.postgresql_replica_2 SELECT 50 + number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_2.postgresql_replica_3 SELECT 50 + number, number from numbers(50) on instance Executing query INSERT INTO postgres_database_2.postgresql_replica_4 SELECT 50 + number, number from numbers(50) on instance Executing query SELECT * FROM `test_database_1`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database_1` WHERE name = 'postgresql_replica_0' on instance Executing query select * from `postgres_database_1`.`postgresql_replica_0` order by key; on instance Executing query select * from `test_database_1`.`postgresql_replica_0` order by key; on instance Executing query SELECT * FROM `test_database_1`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database_1` WHERE name = 'postgresql_replica_1' on instance Executing query select * from `postgres_database_1`.`postgresql_replica_1` order by key; on instance Executing query select * from `test_database_1`.`postgresql_replica_1` order by key; on instance Executing query SELECT * FROM `test_database_1`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database_1` WHERE name = 'postgresql_replica_2' on instance Executing query select * from `postgres_database_1`.`postgresql_replica_2` order by key; on instance Executing query select * from `test_database_1`.`postgresql_replica_2` order by key; on instance Executing query SELECT * FROM `test_database_1`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database_1` WHERE name = 'postgresql_replica_3' on instance Executing query select * from `postgres_database_1`.`postgresql_replica_3` order by key; on instance Executing query select * from `test_database_1`.`postgresql_replica_3` order by key; on instance Stderr: Container roottestreplicatedmutations-gw9-node4-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-node5-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-node2-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-node3-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-node1-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-node5-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-node2-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-node1-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-node4-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-node3-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node3/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node3/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node4/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node4/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Executing query SELECT * FROM `test_database_1`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node5/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node5/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/.env --project-name roottestreplicatedmutations-gw9 --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node3/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node4/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_mutations/_instances-0-gw9/node5/docker-compose.yml down --volumes] Executing query SHOW TABLES FROM `test_database_1` WHERE name = 'postgresql_replica_4' on instance Executing query select * from `postgres_database_1`.`postgresql_replica_4` order by key; on instance Executing query select * from `test_database_1`.`postgresql_replica_4` order by key; on instance Executing query SELECT * FROM `test_database_2`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database_2` WHERE name = 'postgresql_replica_0' on instance Executing query select * from `postgres_database_2`.`postgresql_replica_0` order by key; on instance Executing query select * from `test_database_2`.`postgresql_replica_0` order by key; on instance Stderr: Container roottestreplicatedmutations-gw9-node1-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-node2-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-node4-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-node3-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-node5-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-node1-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-node1-1 Removing Stderr: Container roottestreplicatedmutations-gw9-node2-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-node2-1 Removing Stderr: Container roottestreplicatedmutations-gw9-node4-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-node4-1 Removing Stderr: Container roottestreplicatedmutations-gw9-node3-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-node3-1 Removing Stderr: Container roottestreplicatedmutations-gw9-node5-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-node5-1 Removing Stderr: Container roottestreplicatedmutations-gw9-node5-1 Removed Stderr: Container roottestreplicatedmutations-gw9-node1-1 Removed Stderr: Container roottestreplicatedmutations-gw9-node4-1 Removed Stderr: Container roottestreplicatedmutations-gw9-node3-1 Removed Stderr: Container roottestreplicatedmutations-gw9-node2-1 Removed Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Stopping Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Removing Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Removing Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Stopped Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Removing Stderr: Container roottestreplicatedmutations-gw9-zoo2-1 Removed Stderr: Container roottestreplicatedmutations-gw9-zoo1-1 Removed Stderr: Container roottestreplicatedmutations-gw9-zoo3-1 Removed Stderr: Network roottestreplicatedmutations-gw9_default Removing Stderr: Network roottestreplicatedmutations-gw9_default Removed Cleanup called Docker networks for project roottestreplicatedmutations-gw9 are NETWORK ID NAME DRIVER SCOPE Executing query SELECT * FROM `test_database_2`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Docker containers for project roottestreplicatedmutations-gw9 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedmutations-gw9 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmutations-gw9-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicatedmutations-gw9 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:5 Command:[docker volume prune -f] Executing query SHOW TABLES FROM `test_database_2` WHERE name = 'postgresql_replica_1' on instance Stdout:Total reclaimed space: 0B Volumes pruned: 5 Executing query select * from `postgres_database_2`.`postgresql_replica_1` order by key; on instance Executing query select * from `test_database_2`.`postgresql_replica_1` order by key; on instance Executing query SELECT * FROM `test_database_2`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database_2` WHERE name = 'postgresql_replica_2' on instance Executing query select * from `postgres_database_2`.`postgresql_replica_2` order by key; on instance Executing query select * from `test_database_2`.`postgresql_replica_2` order by key; on instance Executing query SELECT * FROM `test_database_2`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database_2` WHERE name = 'postgresql_replica_3' on instance Executing query select * from `postgres_database_2`.`postgresql_replica_3` order by key; on instance Executing query select * from `test_database_2`.`postgresql_replica_3` order by key; on instance Executing query SELECT * FROM `test_database_2`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database_2` WHERE name = 'postgresql_replica_4' on instance Executing query select * from `postgres_database_2`.`postgresql_replica_4` order by key; on instance Executing query select * from `test_database_2`.`postgresql_replica_4` order by key; on instance Executing query DROP DATABASE IF EXISTS `test_database_1` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_multiple_databases Executing query DROP DATABASE IF EXISTS `test_database_2` SYNC on instance Executing query DROP DATABASE IF EXISTS "postgres_database_1" on instance Executing query DROP DATABASE IF EXISTS "postgres_database_2" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_quoting_1 Executing query INSERT INTO `postgres_database`.`user` SELECT number, number from numbers(50) on instance Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SHOW DATABASES on instance No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/60f72101807a853e670f2496c8666f3e1eac5569940a59b1df3155d25fe1ed02/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/60f72101807a853e670f2496c8666f3e1eac5569940a59b1df3155d25fe1ed02/json HTTP/1.1" 200 587 Executing query SELECT * FROM `test_database`.`user` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'user' on instance Executing query select * from `postgres_database`.`user` order by key; on instance Executing query select * from `test_database`.`user` order by key; on instance Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_quoting_1 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_quoting_2 Executing query INSERT INTO `postgres_database`.`user` SELECT number, number from numbers(50) on instance Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') SETTINGS materialized_postgresql_tables_list = 'user' on instance Executing query SHOW DATABASES on instance Executing query SELECT * FROM `test_database`.`user` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'user' on instance Executing query select * from `postgres_database`.`user` order by key; on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query select * from `test_database`.`user` order by key; on instance Stdout:774 Clickhouse process running. run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:774 Executing query select 20 on backup_node_1 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_quoting_2 run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query DROP DATABASE IF EXISTS "postgres_database" on instance No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/76c551e467792f172de776d17113ffc8070182efc81554de427fc06e260d0c64/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/76c551e467792f172de776d17113ffc8070182efc81554de427fc06e260d0c64/json HTTP/1.1" 200 587 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_replica_identity_index Executing query INSERT INTO postgres_database.postgresql_replica SELECT number, number, number, number from numbers(50, 10) on instance Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Executing query INSERT INTO postgres_database.postgresql_replica SELECT number, number, number, number from numbers(100, 10) on instance Executing query SELECT * FROM `test_database`.`postgresql_replica` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica' on instance Executing query select * from `postgres_database`.`postgresql_replica` order by key1; on instance Executing query select * from `test_database`.`postgresql_replica` order by key1; on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:763 Clickhouse process running. run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:763 Executing query select 20 on backup_node_2 run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-main_node_2-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/b865549f0dec96d56fb626507df663ffe47a8a388dfd4ce475b451f09fc44721/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/b865549f0dec96d56fb626507df663ffe47a8a388dfd4ce475b451f09fc44721/json HTTP/1.1" 200 587 Executing query select * from `test_database`.`postgresql_replica` order by key1; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica' on instance Executing query select * from `postgres_database`.`postgresql_replica` order by key1; on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query select * from `test_database`.`postgresql_replica` order by key1; on instance Stdout:765 Clickhouse process running. run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:765 Executing query select 20 on main_node_2 Executing query SELECT * FROM `test_database`.`postgresql_replica` LIMIT 1 FORMAT Null on instance Executing query SYSTEM SYNC DATABASE REPLICA cluster_groups; on backup_node_1 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica' on instance Executing query SYSTEM SYNC DATABASE REPLICA cluster_groups; on backup_node_2 Executing query select * from `postgres_database`.`postgresql_replica` order by key1; on instance Executing query SYSTEM SYNC DATABASE REPLICA cluster_groups; on main_node_2 Executing query select * from `test_database`.`postgresql_replica` order by key1; on instance Executing query show create table cluster_groups.table_1 on main_node_1 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_replica_identity_index Executing query show create table cluster_groups.table_1 on main_node_2 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query show create table cluster_groups.table_1 on backup_node_1 Executing query show create table cluster_groups.table_1 on backup_node_2 Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query show create table cluster_groups.table_2 on main_node_1 Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query show create table cluster_groups.table_2 on main_node_2 test_postgresql_replica_database_engine_1/test.py::test_replicating_dml Executing query INSERT INTO postgres_database.postgresql_replica_0 SELECT number, 0 from numbers(50) on instance Executing query show create table cluster_groups.table_2 on backup_node_1 Executing query show create table cluster_groups.table_2 on backup_node_2 Executing query INSERT INTO postgres_database.postgresql_replica_1 SELECT number, 1 from numbers(50) on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps -C clickhouse] Executing query INSERT INTO postgres_database.postgresql_replica_2 SELECT number, 2 from numbers(50) on instance Stdout: PID TTY TIME CMD Stdout: 763 ? 00:00:00 clickhouse run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill clickhouse'] Command:[docker exec -u root roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c pkill clickhouse] run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query INSERT INTO postgres_database.postgresql_replica_3 SELECT number, 3 from numbers(50) on instance Stdout:763 Executing query INSERT INTO postgres_database.postgresql_replica_4 SELECT number, 4 from numbers(50) on instance Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Executing query INSERT INTO postgres_database.postgresql_replica_0 SELECT 50 + number, 0 from numbers(1000) on instance Executing query INSERT INTO postgres_database.postgresql_replica_1 SELECT 50 + number, 1 from numbers(1000) on instance Executing query INSERT INTO postgres_database.postgresql_replica_2 SELECT 50 + number, 2 from numbers(1000) on instance Executing query INSERT INTO postgres_database.postgresql_replica_3 SELECT 50 + number, 3 from numbers(1000) on instance Executing query INSERT INTO postgres_database.postgresql_replica_4 SELECT 50 + number, 4 from numbers(1000) on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Stdout:763 Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:763 Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:763 Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SYSTEM DROP DATABASE REPLICA '1|4' FROM DATABASE cluster_groups on backup_node_1 Executing query SELECT host_name from system.clusters WHERE cluster = 'cluster_groups' ORDER BY host_name on backup_node_1 run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps -C clickhouse] Stdout: PID TTY TIME CMD Stdout: 765 ? 00:00:01 clickhouse run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill clickhouse'] Command:[docker exec -u root roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c pkill clickhouse] run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:765 Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:765 Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:765 Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Executing query SYSTEM DROP DATABASE REPLICA '1|2' FROM DATABASE cluster_groups on main_node_1 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance Executing query SELECT host_name from system.clusters WHERE cluster = 'cluster_groups' ORDER BY host_name on main_node_1 Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/0c90399e7b3cbb3177777839edf8eddfdb4f2a98f0e8d7a588c7ea467372b8b3/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/0c90399e7b3cbb3177777839edf8eddfdb4f2a98f0e8d7a588c7ea467372b8b3/json HTTP/1.1" 200 587 Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_replicating_dml Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_restart_server_while_replication_startup_not_finished Executing query INSERT INTO `postgres_database`.postgresql_replica_0 SELECT number, number from numbers(100000) on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:1557 Clickhouse process running. run container_id:roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:1557 Executing query select 20 on backup_node_2 run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query INSERT INTO `postgres_database`.postgresql_replica_1 SELECT number, number from numbers(100000) on instance No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestreplicateddatabaseclustergroups-gw1-main_node_2-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/aacb509d53465a1d5221be9e5ea48cd37deaafacad99179fa3e5852624c61ace/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/aacb509d53465a1d5221be9e5ea48cd37deaafacad99179fa3e5852624c61ace/json HTTP/1.1" 200 587 Executing query INSERT INTO `postgres_database`.postgresql_replica_2 SELECT number, number from numbers(100000) on instance Executing query INSERT INTO `postgres_database`.postgresql_replica_3 SELECT number, number from numbers(100000) on instance Executing query INSERT INTO `postgres_database`.postgresql_replica_4 SELECT number, number from numbers(100000) on instance Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:1547 Clickhouse process running. run container_id:roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:1547 Executing query select 20 on main_node_2 Executing query DROP DATABASE cluster_groups SYNC; on main_node_1 Executing query DROP DATABASE cluster_groups SYNC; on main_node_2 Executing query DROP DATABASE cluster_groups SYNC; on backup_node_1 Executing query DROP DATABASE cluster_groups SYNC; on backup_node_2 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/.env --project-name roottestreplicateddatabaseclustergroups-gw1 --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_1/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_2/docker-compose.yml stop --timeout 20] [gw1] PASSED test_replicated_database_cluster_groups/test.py::test_cluster_groups run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps -C clickhouse] Stdout: PID TTY TIME CMD Stdout: 1587 ? 00:00:11 clickhouse run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill clickhouse'] Command:[docker exec -u root roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c pkill clickhouse] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:1587 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:1587 Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_1-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_1-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/.env --project-name roottestreplicateddatabaseclustergroups-gw1 --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/main_node_2/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_1/docker-compose.yml --file /ClickHouse/tests/integration/test_replicated_database_cluster_groups/_instances-0-gw1/backup_node_2/docker-compose.yml down --volumes] Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_1-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_1-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_1-1 Removing Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 Removing Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 Removing Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 Removing Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_1-1 Removed Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_1-1 Removed Stderr: Container roottestreplicateddatabaseclustergroups-gw1-main_node_2-1 Removed Stderr: Container roottestreplicateddatabaseclustergroups-gw1-backup_node_2-1 Removed Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Stopping Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Removing Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Removing Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Stopped Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Removing Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo2-1 Removed Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo1-1 Removed Stderr: Container roottestreplicateddatabaseclustergroups-gw1-zoo3-1 Removed Stderr: Network roottestreplicateddatabaseclustergroups-gw1_default Removing Stderr: Network roottestreplicateddatabaseclustergroups-gw1_default Removed Cleanup called Docker networks for project roottestreplicateddatabaseclustergroups-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicateddatabaseclustergroups-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicateddatabaseclustergroups-gw1 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicateddatabaseclustergroups-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicateddatabaseclustergroups-gw1 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:5 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 5 test_replicated_merge_tree_compatibility/test.py::test_replicated_merge_tree_defaults_compatibility Running tests in /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/test.py Cluster start called. is_up=False Docker networks for project roottestreplicatedmergetreecompatibility-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedmergetreecompatibility-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedmergetreecompatibility-gw1 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestreplicatedmergetreecompatibility-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedmergetreecompatibility-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedmergetreecompatibility-gw1 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmergetreecompatibility-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Unstopped containers: {} No running containers for project: roottestreplicatedmergetreecompatibility-gw1 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:1587 Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:5 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 5 Setup directory for instance: node1 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files [] to /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node1/database Setup logs dir /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node1/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Setup directory for instance: node2 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files [] to /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node2/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node2/database Setup logs dir /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node2/logs Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ; coproc tail -f /dev/null; wait $$!" Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/.env --project-name roottestreplicatedmergetreecompatibility-gw1 --file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node2/docker-compose.yml pull] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:1587 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestpostgresqlreplicadatabaseengine1-gw0-instance-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/dce0d0e8837e66c5972c2602248188fb7d5bb78096a3f07cc63c983ec041ea94/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/dce0d0e8837e66c5972c2602248188fb7d5bb78096a3f07cc63c983ec041ea94/json HTTP/1.1" 200 587 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:2390 Clickhouse process running. run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:2390 Executing query select 20 on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_restart_server_while_replication_startup_not_finished Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_single_transaction Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Stderr: node1 Skipped - Image is already being pulled by node2 Stderr: zoo3 Skipped - Image is already being pulled by zoo2 Stderr: zoo1 Skipped - Image is already being pulled by zoo2 Stderr: zoo2 Pulling Stderr: node2 Pulling Stderr: node2 Pulled Stderr: zoo2 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper1/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper1/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper1/coordination', '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper2/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper2/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper2/coordination', '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper3/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper3/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/keeper3/coordination'] Command:[docker compose --project-name roottestreplicatedmergetreecompatibility-gw1 --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stderr:time="2025-04-02T03:34:25Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreplicatedmergetreecompatibility-gw1_default Creating Stderr: Network roottestreplicatedmergetreecompatibility-gw1_default Created Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Creating Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Creating Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Creating Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Created Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Created Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Created Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Starting Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Starting Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Starting Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Started Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Started Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Started Stderr:time="2025-04-02T03:34:26Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:34:26Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreecompatibility-gw1-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.1.2, port:2181, use_ssl:False Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query select count() from test_database.postgresql_replica_0 on instance Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_single_transaction Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_table_schema_changes Executing query INSERT INTO postgres_database.postgresql_replica_0 SELECT number, 0, 0, 0 from numbers(25) on instance Executing query INSERT INTO postgres_database.postgresql_replica_1 SELECT number, 1, 1, 1 from numbers(25) on instance Executing query INSERT INTO postgres_database.postgresql_replica_2 SELECT number, 2, 2, 2 from numbers(25) on instance Executing query INSERT INTO postgres_database.postgresql_replica_3 SELECT number, 3, 3, 3 from numbers(25) on instance Executing query INSERT INTO postgres_database.postgresql_replica_4 SELECT number, 4, 4, 4 from numbers(25) on instance Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Executing query INSERT INTO postgres_database.postgresql_replica_0 SELECT 25 + number, 0, 0, 0 from numbers(25) on instance Executing query INSERT INTO postgres_database.postgresql_replica_1 SELECT 25 + number, 1, 1, 1 from numbers(25) on instance Executing query INSERT INTO postgres_database.postgresql_replica_2 SELECT 25 + number, 2, 2, 2 from numbers(25) on instance Executing query INSERT INTO postgres_database.postgresql_replica_3 SELECT 25 + number, 3, 3, 3 from numbers(25) on instance Executing query INSERT INTO postgres_database.postgresql_replica_4 SELECT 25 + number, 4, 4, 4 from numbers(25) on instance Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreecompatibility-gw1-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.1.4, port:2181, use_ssl:False Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreecompatibility-gw1-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.1.3, port:2181, use_ssl:False Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/.env --project-name roottestreplicatedmergetreecompatibility-gw1 --file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node2/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/.env --project-name roottestreplicatedmergetreecompatibility-gw1 --file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node2/docker-compose.yml up -d --no-recreate] Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_1` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_1' on instance Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Running Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Running Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Running Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node2-1 Creating Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node1-1 Creating Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node2-1 Created Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node1-1 Created Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node1-1 Starting Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node2-1 Starting Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node2-1 Started Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node1-1 Started ClickHouse instance created get_instance_ip instance_name=node1 Executing query select * from `postgres_database`.`postgresql_replica_1` order by key; on instance http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreecompatibility-gw1-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreecompatibility-gw1-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.1.6... http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreecompatibility-gw1-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/bb7afbff5d7fd84af7b71c06662f2fa24d07ecb73cc445a95bc8e78cc655bff3/json HTTP/1.1" 200 None Executing query select * from `test_database`.`postgresql_replica_1` order by key; on instance http://localhost:None "GET /v1.46/containers/bb7afbff5d7fd84af7b71c06662f2fa24d07ecb73cc445a95bc8e78cc655bff3/json HTTP/1.1" 200 None Executing query SELECT * FROM `test_database`.`postgresql_replica_2` LIMIT 1 FORMAT Null on instance http://localhost:None "GET /v1.46/containers/bb7afbff5d7fd84af7b71c06662f2fa24d07ecb73cc445a95bc8e78cc655bff3/json HTTP/1.1" 200 None Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_2' on instance Executing query select * from `postgres_database`.`postgresql_replica_2` order by key; on instance http://localhost:None "GET /v1.46/containers/bb7afbff5d7fd84af7b71c06662f2fa24d07ecb73cc445a95bc8e78cc655bff3/json HTTP/1.1" 200 None Executing query select * from `test_database`.`postgresql_replica_2` order by key; on instance http://localhost:None "GET /v1.46/containers/bb7afbff5d7fd84af7b71c06662f2fa24d07ecb73cc445a95bc8e78cc655bff3/json HTTP/1.1" 200 None Executing query SELECT * FROM `test_database`.`postgresql_replica_3` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_3' on instance http://localhost:None "GET /v1.46/containers/bb7afbff5d7fd84af7b71c06662f2fa24d07ecb73cc445a95bc8e78cc655bff3/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreecompatibility-gw1-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreecompatibility-gw1-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.1.5... http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreecompatibility-gw1-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/3eb4fd07f2efca499b372cce58c18ea85dd45859c9e96298f9a13d9741192d3d/json HTTP/1.1" 200 None ClickHouse node2 started Executing query CREATE DATABASE test ENGINE = Ordinary on node1 Executing query CREATE TABLE test.table ( a UInt32, b String DEFAULT If(a = 0, 'true', 'false'), c String DEFAULT Cast(a, 'String') ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/table', 'node1') ORDER BY a on node1 Executing query select * from `postgres_database`.`postgresql_replica_3` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_3` order by key; on instance Executing query CREATE DATABASE test ENGINE = Ordinary on node2 Executing query SELECT * FROM `test_database`.`postgresql_replica_4` LIMIT 1 FORMAT Null on instance Executing query CREATE TABLE test.table ( a UInt32, b String DEFAULT If(a = 0, 'true', 'false'), c String DEFAULT Cast(a, 'String') ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/table', 'node2') ORDER BY a on node2 Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_4' on instance Executing query DETACH TABLE test.table on node1 Executing query select * from `postgres_database`.`postgresql_replica_4` order by key; on instance Executing query SYSTEM DROP REPLICA 'node1' FROM TABLE test.table on node2 Executing query select * from `test_database`.`postgresql_replica_4` order by key; on instance run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'rm /var/lib/clickhouse/metadata/test/table.sql'] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c rm /var/lib/clickhouse/metadata/test/table.sql] Executing query SELECT key, value1, value3 FROM test_database.postgresql_replica_3 ORDER BY key on instance run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'rm -r /var/lib/clickhouse/data/test/table'] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c rm -r /var/lib/clickhouse/data/test/table] get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreecompatibility-gw1-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.1.2, port:2181, use_ssl:False Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): Exists(path='/clickhouse/tables/test/table/replicas/node1', watcher=None) run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -15 clickhouse'] Command:[docker exec -u root roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c pkill -15 clickhouse] Executing query SELECT count() FROM test_database.postgresql_replica_0 on instance run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:8 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'timeout 30 tail -Fn100 "/var/log/clickhouse-server/clickhouse-server.log" | grep -Em 1 \'Table postgresql_replica_0 is skipped from replication stream\''] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c timeout 30 tail -Fn100 "/var/log/clickhouse-server/clickhouse-server.log" | grep -Em 1 'Table postgresql_replica_0 is skipped from replication stream'] run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:8 run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:2025.04.02 03:34:32.641083 [ 2734 ] {} PostgreSQLReplicaConsumer(postgres_database): Table postgresql_replica_0 is skipped from replication stream because its structure has changes. Please detach this table and reattach to resume the replication (relation id: 16906) 1 log line(s) matching "Table postgresql_replica_0 is skipped from replication stream" appeared in a 1.018 seconds Executing query SELECT count() FROM test_database.postgresql_replica_0 on instance Stdout:8 Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_table_schema_changes Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_user_managed_slots Executing query INSERT INTO `postgres_database`.`test_table` SELECT number, number from numbers(50) on instance run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Executing query DROP DATABASE IF EXISTS `test_database` on instance run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'cp /usr/bin/clickhouse /usr/share/clickhouse_original'] Command:[docker exec -u root roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c cp /usr/bin/clickhouse /usr/share/clickhouse_original] Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') SETTINGS materialized_postgresql_replication_slot = 'user_slot', materialized_postgresql_snapshot = '000000DC-0000000E-1' on instance Executing query SHOW DATABASES on instance Executing query SELECT * FROM `test_database`.`test_table` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_table' on instance Executing query select * from `postgres_database`.`test_table` order by key; on instance Executing query select * from `test_database`.`test_table` order by key; on instance run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'cp /usr/share/clickhouse_fresh /usr/bin/clickhouse && chmod 777 /usr/bin/clickhouse'] Command:[docker exec -u root roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c cp /usr/share/clickhouse_fresh /usr/bin/clickhouse && chmod 777 /usr/bin/clickhouse] Executing query INSERT INTO postgres_database.test_table SELECT number, number from numbers(10000, 10000) on instance Executing query SELECT * FROM `test_database`.`test_table` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_table' on instance Executing query select * from `postgres_database`.`test_table` order by key; on instance Executing query select * from `test_database`.`test_table` order by key; on instance run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "echo 'restart_with_latest_version: From version' && /usr/share/clickhouse_original server --version && echo 'To version' /usr/share/clickhouse_fresh server --version"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c echo 'restart_with_latest_version: From version' && /usr/share/clickhouse_original server --version && echo 'To version' /usr/share/clickhouse_fresh server --version] Stdout:restart_with_latest_version: From version Stdout:ClickHouse server version 23.3.22.3 (official build). Stdout:To version /usr/share/clickhouse_fresh server --version run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- '] Command:[docker exec -u 0 roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ] run container_id:roottestreplicatedmergetreecompatibility-gw1-node1-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node1-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:393 Executing query select 20 on node1 Executing query select * from `test_database`.`test_table` order by key; on instance run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:True cmd: ['bash', '-c', 'ps -C clickhouse'] Command:[docker exec -u root roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps -C clickhouse] Executing query select 20 on node1 Stdout: PID TTY TIME CMD Stdout: 2390 ? 00:00:03 clickhouse run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill clickhouse'] Command:[docker exec -u root roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c pkill clickhouse] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'pkill -15 clickhouse'] Command:[docker exec -u root roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c pkill -15 clickhouse] Stdout:2390 run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:8 run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:8 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:2390 run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:8 run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c ps ax | grep 'clickhouse server' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'cp /usr/bin/clickhouse /usr/share/clickhouse_original'] Command:[docker exec -u root roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c cp /usr/bin/clickhouse /usr/share/clickhouse_original] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:2390 run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'cp /usr/share/clickhouse_fresh /usr/bin/clickhouse && chmod 777 /usr/bin/clickhouse'] Command:[docker exec -u root roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c cp /usr/share/clickhouse_fresh /usr/bin/clickhouse && chmod 777 /usr/bin/clickhouse] run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', "echo 'restart_with_latest_version: From version' && /usr/share/clickhouse_original server --version && echo 'To version' /usr/share/clickhouse_fresh server --version"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c echo 'restart_with_latest_version: From version' && /usr/share/clickhouse_original server --version && echo 'To version' /usr/share/clickhouse_fresh server --version] Stdout:restart_with_latest_version: From version Stdout:ClickHouse server version 23.3.22.3 (official build). Stdout:To version /usr/share/clickhouse_fresh server --version run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- '] Command:[docker exec -u 0 roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon -- ] run container_id:roottestreplicatedmergetreecompatibility-gw1-node2-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestreplicatedmergetreecompatibility-gw1-node2-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:379 Executing query select 20 on node2 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:2390 Executing query select 20 on node2 Executing query CREATE TABLE test.table ( a UInt32, b String DEFAULT If(a = 0, 'true', 'false'), c String DEFAULT Cast(a, 'String') ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/test/table', '1') ORDER BY a on node1 Executing query EXISTS TABLE test.table on node1 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/.env --project-name roottestreplicatedmergetreecompatibility-gw1 --file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node2/docker-compose.yml stop --timeout 20] [gw1] PASSED test_replicated_merge_tree_compatibility/test.py::test_replicated_merge_tree_defaults_compatibility run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] No clickhouse process running. Start new one. http://localhost:None "POST /v1.46/containers/roottestpostgresqlreplicadatabaseengine1-gw0-instance-1/exec HTTP/1.1" 201 74 http://localhost:None "POST /v1.46/exec/3a7f85acf2e61bcc42378c8d704cd57a061cd529dc4407bd01a3220fce57e601/start HTTP/1.1" 200 0 http://localhost:None "GET /v1.46/exec/3a7f85acf2e61bcc42378c8d704cd57a061cd529dc4407bd01a3220fce57e601/json HTTP/1.1" 200 587 run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:3202 Clickhouse process running. run container_id:roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 detach:False nothrow:False cmd: ['bash', '-c', "ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'"] Command:[docker exec roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 bash -c ps ax | grep 'clickhouse' | grep -v 'grep' | grep -v 'coproc' | grep -v 'bash -c' | awk '{print $1}'] Stdout:3202 Executing query select 20 on instance Executing query INSERT INTO postgres_database.test_table SELECT number, number from numbers(20000, 10000) on instance Executing query SELECT * FROM `test_database`.`test_table` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'test_table' on instance Executing query select * from `postgres_database`.`test_table` order by key; on instance Executing query select * from `test_database`.`test_table` order by key; on instance Executing query select * from `test_database`.`test_table` order by key; on instance Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_user_managed_slots Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance test_postgresql_replica_database_engine_1/test.py::test_virtual_columns Executing query DROP DATABASE IF EXISTS `test_database` on instance Executing query CREATE DATABASE `test_database` ENGINE = MaterializedPostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Executing query SHOW DATABASES on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Executing query INSERT INTO postgres_database.postgresql_replica_0 SELECT number, number from numbers(10) on instance Executing query SELECT * FROM `test_database`.`postgresql_replica_0` LIMIT 1 FORMAT Null on instance Executing query SHOW TABLES FROM `test_database` WHERE name = 'postgresql_replica_0' on instance Executing query select * from `postgres_database`.`postgresql_replica_0` order by key; on instance Executing query select * from `test_database`.`postgresql_replica_0` order by key; on instance Executing query SELECT key, value, _sign, _version FROM test_database.postgresql_replica_0; on instance Executing query DROP DATABASE IF EXISTS `test_database` SYNC on instance [gw0] PASSED test_postgresql_replica_database_engine_1/test.py::test_virtual_columns Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query DROP DATABASE IF EXISTS "postgres_database" on instance Executing query CREATE DATABASE "postgres_database" ENGINE = PostgreSQL('172.16.10.2:5432', 'postgres_database', 'postgres', 'mysecretpassword') on instance Command:[docker compose --env-file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/.env --project-name roottestpostgresqlreplicadatabaseengine1-gw0 --file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_postgres.yml stop --timeout 20] Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 Stopping Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Stopping Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 Stopped Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/.env --project-name roottestpostgresqlreplicadatabaseengine1-gw0 --file /ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/_instances-0-gw0/instance/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_postgres.yml down --volumes] Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 Stopping Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Stopping Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 Stopped Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 Removing Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Stopped Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Removing Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-instance-1 Removed Stderr: Container roottestpostgresqlreplicadatabaseengine1-gw0-postgres1-1 Removed Stderr: Network roottestpostgresqlreplicadatabaseengine1-gw0_default Removing Stderr: Network roottestpostgresqlreplicadatabaseengine1-gw0_default Removed Cleanup called Docker networks for project roottestpostgresqlreplicadatabaseengine1-gw0 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestpostgresqlreplicadatabaseengine1-gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestpostgresqlreplicadatabaseengine1-gw0 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestpostgresqlreplicadatabaseengine1-gw0-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestpostgresqlreplicadatabaseengine1-gw0 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:4 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 4 Connection dropped: outstanding heartbeat ping not received Transition to CONNECTING Zookeeper connection lost Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: No route to host Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Connection dropped: socket connection error: No route to host Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node2-1 Stopping Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node1-1 Stopping Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node1-1 Stopped Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node2-1 Stopped Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Stopping Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Stopping Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Stopping Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Stopped Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Stopped Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/.env --project-name roottestreplicatedmergetreecompatibility-gw1 --file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_compatibility/_instances-0-gw1/node2/docker-compose.yml down --volumes] Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node1-1 Stopping Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node2-1 Stopping Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node1-1 Stopped Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node1-1 Removing Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node2-1 Stopped Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node2-1 Removing Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node2-1 Removed Stderr: Container roottestreplicatedmergetreecompatibility-gw1-node1-1 Removed Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Stopping Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Stopping Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Stopping Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Stopped Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Removing Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Stopped Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Removing Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Stopped Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Removing Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo3-1 Removed Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo1-1 Removed Stderr: Container roottestreplicatedmergetreecompatibility-gw1-zoo2-1 Removed Stderr: Network roottestreplicatedmergetreecompatibility-gw1_default Removing Stderr: Network roottestreplicatedmergetreecompatibility-gw1_default Removed Cleanup called Docker networks for project roottestreplicatedmergetreecompatibility-gw1 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedmergetreecompatibility-gw1 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedmergetreecompatibility-gw1 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmergetreecompatibility-gw1-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicatedmergetreecompatibility-gw1 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:4 Command:[docker volume prune -f] Stdout:Total reclaimed space: 0B Volumes pruned: 4 Connection dropped: socket connection error: None Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --project-name roottestreloadclientcertificate-gw4 --file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml --file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node2/docker-compose.yml stop --timeout 20] [gw4] PASSED test_reload_client_certificate/test.py::test_wrong_cn_cert Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-node1-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-node2-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopped Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopped Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopped Stderr: Container roottestreloadclientcertificate-gw4-node2-1 Stopped Stderr: Container roottestreloadclientcertificate-gw4-node1-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/.env --project-name roottestreloadclientcertificate-gw4 --file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_zookeeper_secure.yml --file /ClickHouse/tests/integration/test_reload_client_certificate/_instances-0-gw4/node2/docker-compose.yml down --volumes] Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-node1-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-node2-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopping Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Stopped Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Removing Stderr: Container roottestreloadclientcertificate-gw4-node1-1 Stopped Stderr: Container roottestreloadclientcertificate-gw4-node1-1 Removing Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Stopped Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Removing Stderr: Container roottestreloadclientcertificate-gw4-node2-1 Stopped Stderr: Container roottestreloadclientcertificate-gw4-node2-1 Removing Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Stopped Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Removing Stderr: Container roottestreloadclientcertificate-gw4-node2-1 Removed Stderr: Container roottestreloadclientcertificate-gw4-node1-1 Removed Stderr: Container roottestreloadclientcertificate-gw4-zoo3-1 Removed Stderr: Container roottestreloadclientcertificate-gw4-zoo2-1 Removed Stderr: Container roottestreloadclientcertificate-gw4-zoo1-1 Removed Stderr: Network roottestreloadclientcertificate-gw4_default Removing Stderr: Network roottestreloadclientcertificate-gw4_default Removed Cleanup called Docker networks for project roottestreloadclientcertificate-gw4 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreloadclientcertificate-gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreloadclientcertificate-gw4 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreloadclientcertificate-gw4-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreloadclientcertificate-gw4 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:1 Volumes pruned: 1 test_replicated_merge_tree_encrypted_disk/test.py::test_different_keys Running tests in /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/test.py Cluster start called. is_up=False Docker networks for project roottestreplicatedmergetreeencrypteddisk-gw4 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedmergetreeencrypteddisk-gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedmergetreeencrypteddisk-gw4 are DRIVER VOLUME NAME Cleanup called Docker networks for project roottestreplicatedmergetreeencrypteddisk-gw4 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedmergetreeencrypteddisk-gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedmergetreeencrypteddisk-gw4 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmergetreeencrypteddisk-gw4-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicatedmergetreeencrypteddisk-gw4 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:1 Volumes pruned: 1 Setup directory for instance: node1 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/configs/storage.xml'] to /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node1/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node1/database Setup logs dir /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node1/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Setup directory for instance: node2 Create directory for configuration generated in this helper Create directory for common tests configuration Copy common configuration from helpers Generate and write macros file Copy custom test config files ['/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/configs/remote_servers.xml', '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/configs/storage.xml'] to /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node2/configs/config.d Setup database dir /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node2/database Setup logs dir /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node2/logs Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log", "--"] Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:8b2301119731', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/.env Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] No config file found http://localhost:None "GET /version HTTP/1.1" 200 826 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/.env --project-name roottestreplicatedmergetreeencrypteddisk-gw4 --file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node2/docker-compose.yml pull] Connection dropped: socket connection error: None Stderr: zoo2 Skipped - Image is already being pulled by zoo1 Stderr: zoo3 Skipped - Image is already being pulled by zoo1 Stderr: node2 Skipped - Image is already being pulled by zoo1 Stderr: node1 Skipped - Image is already being pulled by zoo1 Stderr: zoo1 Pulling Stderr: zoo1 Pulled Setup ZooKeeper Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper1/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper1/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper1/coordination', '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper2/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper2/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper2/coordination', '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper3/log', '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper3/config', '/ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/keeper3/coordination'] Command:[docker compose --project-name roottestreplicatedmergetreeencrypteddisk-gw4 --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/.env --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --verbose up -d] Stderr:time="2025-04-02T03:35:56Z" level=trace msg="Docker Desktop integration not enabled" Stderr: Network roottestreplicatedmergetreeencrypteddisk-gw4_default Creating Stderr: Network roottestreplicatedmergetreeencrypteddisk-gw4_default Created Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Creating Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Creating Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Creating Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Created Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Created Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Created Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Starting Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Starting Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Starting Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Started Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Started Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Started Stderr:time="2025-04-02T03:35:57Z" level=debug msg="otel error" error="" Stderr:time="2025-04-02T03:35:57Z" level=debug msg="otel error" error="" Wait ZooKeeper to start get_instance_ip instance_name=zoo1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1/json HTTP/1.1" 200 None get_kazoo_client: zoo1, ip:172.16.1.3, port:2181, use_ssl:False Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Connection dropped: socket connection error: Connection refused Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1/json HTTP/1.1" 200 None get_kazoo_client: zoo2, ip:172.16.1.4, port:2181, use_ssl:False Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED get_instance_ip instance_name=zoo3 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1/json HTTP/1.1" 200 None get_kazoo_client: zoo3, ip:172.16.1.2, port:2181, use_ssl:False Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=30000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) Zookeeper connection established, state: CONNECTED Sending request(xid=1): GetChildren(path='/', watcher=None) Received response(xid=1): ['keeper'] Sending request(xid=2): Close() Connection dropped: socket connection broken Transition to CONNECTING Zookeeper connection lost Failed connecting to Zookeeper within the connection retry policy. Zookeeper session closed, state: CLOSED All instances of ZooKeeper started: ('zoo1', 'zoo2', 'zoo3') ('Trying to create ClickHouse instance by command %s', 'docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/.env --project-name roottestreplicatedmergetreeencrypteddisk-gw4 --file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node2/docker-compose.yml up -d --no-recreate') Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/.env --project-name roottestreplicatedmergetreeencrypteddisk-gw4 --file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node2/docker-compose.yml up -d --no-recreate] Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Running Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Running Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Running Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 Creating Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 Creating Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 Created Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 Created Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 Starting Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 Starting Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 Started Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 Started ClickHouse instance created get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencrypteddisk-gw4-node1-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node1 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencrypteddisk-gw4-node1-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node1, ip: 172.16.1.6... http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencrypteddisk-gw4-node1-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/f9f4b78240cf03f3ff8108b156181d48ad251acc38294380bada5223562d9fd3/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/f9f4b78240cf03f3ff8108b156181d48ad251acc38294380bada5223562d9fd3/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/f9f4b78240cf03f3ff8108b156181d48ad251acc38294380bada5223562d9fd3/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/f9f4b78240cf03f3ff8108b156181d48ad251acc38294380bada5223562d9fd3/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/f9f4b78240cf03f3ff8108b156181d48ad251acc38294380bada5223562d9fd3/json HTTP/1.1" 200 None ClickHouse node1 started get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencrypteddisk-gw4-node2-1/json HTTP/1.1" 200 None get_instance_ip instance_name=node2 http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencrypteddisk-gw4-node2-1/json HTTP/1.1" 200 None Waiting for ClickHouse start in node2, ip: 172.16.1.5... http://localhost:None "GET /v1.46/containers/roottestreplicatedmergetreeencrypteddisk-gw4-node2-1/json HTTP/1.1" 200 None http://localhost:None "GET /v1.46/containers/ee692782f4d74bde9bb480d32bbc0fe9a5b1cec6a117a3f6da18583ff3cc75d7/json HTTP/1.1" 200 None ClickHouse node2 started run container_id:roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8c3RvcmFnZV9jb25maWd1cmF0aW9uPgogICAgICAgIDxkaXNrcz4KICAgICAgICAgICAgPGRpc2tfZW5jcnlwdGVkPgogICAgICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgICAgIDwvZGlza19lbmNyeXB0ZWQ+CiAgICAgICAgPC9kaXNrcz4KICAgIDwvc3RvcmFnZV9jb25maWd1cmF0aW9uPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8c3RvcmFnZV9jb25maWd1cmF0aW9uPgogICAgICAgIDxkaXNrcz4KICAgICAgICAgICAgPGRpc2tfZW5jcnlwdGVkPgogICAgICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgICAgIDwvZGlza19lbmNyeXB0ZWQ+CiAgICAgICAgPC9kaXNrcz4KICAgIDwvc3RvcmFnZV9jb25maWd1cmF0aW9uPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] Executing query SYSTEM RELOAD CONFIG on node1 run container_id:roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8c3RvcmFnZV9jb25maWd1cmF0aW9uPgogICAgICAgIDxkaXNrcz4KICAgICAgICAgICAgPGRpc2tfZW5jcnlwdGVkPgogICAgICAgICAgICAgICAgPGtleT5iYmJiYmJiYmJiYmJiYmJiPC9rZXk+CiAgICAgICAgICAgIDwvZGlza19lbmNyeXB0ZWQ+CiAgICAgICAgPC9kaXNrcz4KICAgIDwvc3RvcmFnZV9jb25maWd1cmF0aW9uPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8c3RvcmFnZV9jb25maWd1cmF0aW9uPgogICAgICAgIDxkaXNrcz4KICAgICAgICAgICAgPGRpc2tfZW5jcnlwdGVkPgogICAgICAgICAgICAgICAgPGtleT5iYmJiYmJiYmJiYmJiYmJiPC9rZXk+CiAgICAgICAgICAgIDwvZGlza19lbmNyeXB0ZWQ+CiAgICAgICAgPC9kaXNrcz4KICAgIDwvc3RvcmFnZV9jb25maWd1cmF0aW9uPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] Executing query SYSTEM RELOAD CONFIG on node2 Executing query DROP TABLE IF EXISTS tbl ON CLUSTER 'cluster' SYNC on node1 Executing query CREATE TABLE tbl ON CLUSTER 'cluster' ( id Int64, str String ) ENGINE=ReplicatedMergeTree('/clickhouse/tables/tbl/', '{replica}') ORDER BY id SETTINGS storage_policy='encrypted_policy' on node1 Executing query INSERT INTO tbl VALUES (1, 'str1') on node1 Executing query INSERT INTO tbl VALUES (1, 'str1') on node2 Executing query INSERT INTO tbl VALUES (2, 'str2') on node2 Executing query SYSTEM SYNC REPLICA tbl LIGHTWEIGHT on node1 Executing query SYSTEM SYNC REPLICA tbl LIGHTWEIGHT on node2 Executing query SELECT * FROM tbl ORDER BY id on node1 Executing query SELECT * FROM tbl ORDER BY id on node2 Executing query CHECK TABLE tbl on node1 Executing query CHECK TABLE tbl on node2 Executing query OPTIMIZE TABLE tbl ON CLUSTER 'cluster' FINAL on node1 Executing query SYSTEM SYNC REPLICA tbl LIGHTWEIGHT on node1 Executing query SYSTEM SYNC REPLICA tbl LIGHTWEIGHT on node2 Executing query SELECT * FROM tbl ORDER BY id on node1 Executing query SELECT * FROM tbl ORDER BY id on node2 Executing query CHECK TABLE tbl on node1 Executing query CHECK TABLE tbl on node2 [gw4] PASSED test_replicated_merge_tree_encrypted_disk/test.py::test_different_keys test_replicated_merge_tree_encrypted_disk/test.py::test_same_keys run container_id:roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8c3RvcmFnZV9jb25maWd1cmF0aW9uPgogICAgICAgIDxkaXNrcz4KICAgICAgICAgICAgPGRpc2tfZW5jcnlwdGVkPgogICAgICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgICAgIDwvZGlza19lbmNyeXB0ZWQ+CiAgICAgICAgPC9kaXNrcz4KICAgIDwvc3RvcmFnZV9jb25maWd1cmF0aW9uPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8c3RvcmFnZV9jb25maWd1cmF0aW9uPgogICAgICAgIDxkaXNrcz4KICAgICAgICAgICAgPGRpc2tfZW5jcnlwdGVkPgogICAgICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgICAgIDwvZGlza19lbmNyeXB0ZWQ+CiAgICAgICAgPC9kaXNrcz4KICAgIDwvc3RvcmFnZV9jb25maWd1cmF0aW9uPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] Executing query SYSTEM RELOAD CONFIG on node1 run container_id:roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 detach:False nothrow:False cmd: ['bash', '-c', 'mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8c3RvcmFnZV9jb25maWd1cmF0aW9uPgogICAgICAgIDxkaXNrcz4KICAgICAgICAgICAgPGRpc2tfZW5jcnlwdGVkPgogICAgICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgICAgIDwvZGlza19lbmNyeXB0ZWQ+CiAgICAgICAgPC9kaXNrcz4KICAgIDwvc3RvcmFnZV9jb25maWd1cmF0aW9uPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml'] Command:[docker exec roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 bash -c mkdir -p $(dirname /etc/clickhouse-server/config.d/z_keys.xml) && echo PGNsaWNraG91c2U+CiAgICA8c3RvcmFnZV9jb25maWd1cmF0aW9uPgogICAgICAgIDxkaXNrcz4KICAgICAgICAgICAgPGRpc2tfZW5jcnlwdGVkPgogICAgICAgICAgICAgICAgPGtleT5hYWFhYWFhYWFhYWFhYWFhPC9rZXk+CiAgICAgICAgICAgIDwvZGlza19lbmNyeXB0ZWQ+CiAgICAgICAgPC9kaXNrcz4KICAgIDwvc3RvcmFnZV9jb25maWd1cmF0aW9uPgo8L2NsaWNraG91c2U+Cg== | base64 --decode > /etc/clickhouse-server/config.d/z_keys.xml] Executing query SYSTEM RELOAD CONFIG on node2 Executing query DROP TABLE IF EXISTS tbl ON CLUSTER 'cluster' SYNC on node1 Executing query CREATE TABLE tbl ON CLUSTER 'cluster' ( id Int64, str String ) ENGINE=ReplicatedMergeTree('/clickhouse/tables/tbl/', '{replica}') ORDER BY id SETTINGS storage_policy='encrypted_policy' on node1 Executing query INSERT INTO tbl VALUES (1, 'str1') on node1 Executing query INSERT INTO tbl VALUES (1, 'str1') on node2 Executing query INSERT INTO tbl VALUES (2, 'str2') on node2 Executing query SYSTEM SYNC REPLICA tbl LIGHTWEIGHT on node1 Executing query SYSTEM SYNC REPLICA tbl LIGHTWEIGHT on node2 Executing query SELECT * FROM tbl ORDER BY id on node1 Executing query SELECT * FROM tbl ORDER BY id on node2 Executing query CHECK TABLE tbl on node1 Executing query CHECK TABLE tbl on node2 Executing query OPTIMIZE TABLE tbl ON CLUSTER 'cluster' FINAL on node1 Executing query SYSTEM SYNC REPLICA tbl LIGHTWEIGHT on node1 Executing query SYSTEM SYNC REPLICA tbl LIGHTWEIGHT on node2 Executing query SELECT * FROM tbl ORDER BY id on node1 Executing query SELECT * FROM tbl ORDER BY id on node2 Executing query CHECK TABLE tbl on node1 Executing query CHECK TABLE tbl on node2 Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/.env --project-name roottestreplicatedmergetreeencrypteddisk-gw4 --file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node2/docker-compose.yml stop --timeout 20] [gw4] PASSED test_replicated_merge_tree_encrypted_disk/test.py::test_same_keys Connection dropped: outstanding heartbeat ping not received Transition to CONNECTING Connection dropped: socket connection error: No route to host Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 Stopping Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 Stopping Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 Stopped Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 Stopped Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Stopping Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Stopping Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Stopping Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Stopped Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Stopped Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Stopped Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node1/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[bash -c [ -f /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true] Command:[docker compose --env-file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/.env --project-name roottestreplicatedmergetreeencrypteddisk-gw4 --file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node1/docker-compose.yml --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_replicated_merge_tree_encrypted_disk/_instances-0-gw4/node2/docker-compose.yml down --volumes] Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 Stopping Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 Stopping Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 Stopped Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 Removing Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 Stopped Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 Removing Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node1-1 Removed Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-node2-1 Removed Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Stopping Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Stopping Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Stopping Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Stopped Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Removing Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Stopped Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Removing Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Stopped Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Removing Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo1-1 Removed Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo3-1 Removed Stderr: Container roottestreplicatedmergetreeencrypteddisk-gw4-zoo2-1 Removed Stderr: Network roottestreplicatedmergetreeencrypteddisk-gw4_default Removing Stderr: Network roottestreplicatedmergetreeencrypteddisk-gw4_default Removed Cleanup called Docker networks for project roottestreplicatedmergetreeencrypteddisk-gw4 are NETWORK ID NAME DRIVER SCOPE Docker containers for project roottestreplicatedmergetreeencrypteddisk-gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Docker volumes for project roottestreplicatedmergetreeencrypteddisk-gw4 are DRIVER VOLUME NAME Command:[docker container list --all --filter name='^/roottestreplicatedmergetreeencrypteddisk-gw4-.*-1$' --format '{{.ID}}:{{.Names}}'] Unstopped containers: {} No running containers for project: roottestreplicatedmergetreeencrypteddisk-gw4 Trying to prune unused networks... Trying to prune unused images... Command:[docker image prune -f] Stdout:Total reclaimed space: 0B Images pruned Trying to prune unused volumes... Command:[docker volume ls | wc -l] Stdout:1 Volumes pruned: 1 =============================== warnings summary =============================== test_reload_client_certificate/test.py: 42 warnings /usr/local/lib/python3.10/dist-packages/kazoo/handlers/utils.py:225: DeprecationWarning: ssl.PROTOCOL_TLS is deprecated context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) test_postgresql_replica_database_engine_1/test.py::test_many_concurrent_queries /usr/local/lib/python3.10/dist-packages/_pytest/threadexception.py:73: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-36 (attack) Traceback (most recent call last): File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/usr/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/ClickHouse/tests/integration/test_postgresql_replica_database_engine_1/test.py", line 429, in attack cursor.execute(query_pool[query_id].format(random_table_name)) psycopg2.errors.NumericValueOutOfRange: integer out of range warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg)) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ============================== slowest durations =============================== 182.41s call test_reload_client_certificate/test.py::test_wrong_cn_cert 44.41s call test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes0] 43.12s call test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes1] 42.45s call test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper 39.52s call test_replicated_merge_tree_s3_zero_copy/test.py::test_drop_table 32.59s call test_replicated_database_cluster_groups/test.py::test_cluster_groups 32.56s call test_replicated_mutations/test.py::test_mutations 29.00s call test_reload_zookeeper/test.py::test_reload_zookeeper 26.07s call test_postgresql_replica_database_engine_1/test.py::test_abrupt_connection_loss_while_heavy_replication 24.52s teardown test_replicated_merge_tree_encrypted_disk/test.py::test_same_keys 24.06s call test_reload_client_certificate/test.py::test_correct_cn_cert 23.89s call test_postgresql_replica_database_engine_1/test.py::test_abrupt_server_restart_while_heavy_replication 22.63s teardown test_replicated_merge_tree_s3_zero_copy/test.py::test_s3_check_restore 21.70s teardown test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper 20.75s setup test_replicated_merge_tree_s3_zero_copy/test.py::test_drop_table 20.58s setup test_replicated_engine_arguments/test.py::test_replicated_engine_with_arguments 20.55s call test_rename_column/test.py::test_rename_with_parallel_ttl_delete 20.49s setup test_postgresql_replica_database_engine_1/test.py::test_abrupt_connection_loss_while_heavy_replication 20.03s setup test_reload_client_certificate/test.py::test_correct_cn_cert 19.90s call test_postgresql_replica_database_engine_1/test.py::test_concurrent_transactions 19.28s call test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch 18.62s setup test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings 18.34s setup test_reload_zookeeper/test.py::test_reload_zookeeper 18.19s setup test_replicated_database_alter_modify_order_by/test.py::test_alter_modify_order_by 18.19s setup test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids 18.18s setup test_reloading_storage_configuration/test.py::test_add_disk 18.08s setup test_replicated_mutations/test.py::test_mutations 17.26s setup test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table 17.23s call test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select 17.12s call test_rename_column/test.py::test_rename_with_parallel_slow_insert 16.97s setup test_rename_column/test.py::test_rename_distributed 16.78s setup test_replicated_merge_tree_encrypted_disk/test.py::test_different_keys 16.51s setup test_replicated_merge_tree_compatibility/test.py::test_replicated_merge_tree_defaults_compatibility 16.19s setup test_replicated_database_cluster_groups/test.py::test_cluster_groups 15.63s setup test_quota/test.py::test_add_remove_interval 15.61s teardown test_replicated_merge_tree_compatibility/test.py::test_replicated_merge_tree_defaults_compatibility 15.21s teardown test_reload_client_certificate/test.py::test_wrong_cn_cert 14.73s call test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server 14.20s call test_rename_column/test.py::test_rename_with_parallel_ttl_move 13.86s setup test_partition/test.py::test_attach_check_all_parts 13.56s call test_rename_column/test.py::test_rename_with_parallel_merges 12.62s setup test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper 12.30s setup test_part_log_table/test.py::test_config_disk_name_test 12.05s setup test_reload_query_masking_rules/test.py::test_reload_query_masking_rules 11.60s teardown test_replicated_merge_tree_encryption_codec/test.py::test_same_keys 11.38s teardown test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes1] 11.30s call test_postgresql_replica_database_engine_1/test.py::test_many_concurrent_queries 11.17s call test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server 10.87s setup test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop 10.84s call test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table 10.64s setup test_remote_prewhere/test.py::test_remote 10.56s call test_postgresql_replica_database_engine_1/test.py::test_drop_database_while_replication_startup_not_finished 10.09s call test_postgresql_replica_database_engine_1/test.py::test_restart_server_while_replication_startup_not_finished 9.60s call test_postgresql_replica_database_engine_1/test.py::test_user_managed_slots 9.35s call test_postgresql_replica_database_engine_1/test.py::test_clickhouse_restart 9.11s call test_rename_column/test.py::test_rename_with_parallel_insert 8.99s call test_postgresql_replica_database_engine_1/test.py::test_replicating_dml 8.95s call test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table 8.23s call test_replicated_merge_tree_compatibility/test.py::test_replicated_merge_tree_defaults_compatibility 7.95s teardown test_rename_column/test.py::test_rename_with_parallel_ttl_move 7.82s teardown test_reload_zookeeper/test.py::test_reload_zookeeper 7.61s call test_postgresql_replica_database_engine_1/test.py::test_single_transaction 7.50s call test_rename_column/test.py::test_rename_distributed 6.52s call test_rename_column/test.py::test_rename_parallel 6.18s call test_postgresql_replica_database_engine_1/test.py::test_multiple_databases 5.93s call test_rename_column/test.py::test_rename_parallel_same_node 5.88s teardown test_quota/test.py::test_users_xml_is_readonly 5.74s teardown test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch 5.65s call test_postgresql_replica_database_engine_1/test.py::test_different_data_types 5.45s call test_rename_column/test.py::test_rename_with_parallel_select 5.41s call test_partition/test.py::test_system_detached_parts 5.36s call test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_subset_of_database_tables 5.34s call test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop 5.23s call test_reloading_storage_configuration/test.py::test_add_policy 5.19s call test_reloading_storage_configuration/test.py::test_remove_disk_from_policy 5.10s teardown test_remote_prewhere/test.py::test_remote 5.09s call test_reloading_storage_configuration/test.py::test_add_disk_to_policy 5.05s call test_reloading_storage_configuration/test.py::test_remove_volume_from_policy 5.03s teardown test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop 4.99s call test_reloading_storage_configuration/test.py::test_new_policy_works 4.98s teardown test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings 4.95s call test_reloading_storage_configuration/test.py::test_remove_disk 4.93s call test_reloading_storage_configuration/test.py::test_add_disk_to_separate_config 4.93s call test_reloading_storage_configuration/test.py::test_add_volume_to_policy 4.91s call test_reloading_storage_configuration/test.py::test_remove_policy 4.90s call test_reloading_storage_configuration/test.py::test_update_disk 4.75s call test_reloading_storage_configuration/test.py::test_add_disk 4.69s teardown test_part_log_table/test.py::test_part_log_contains_partition 4.65s teardown test_partition/test.py::test_system_detached_parts 4.47s call test_postgresql_replica_database_engine_1/test.py::test_table_schema_changes 4.32s teardown test_replicated_engine_arguments/test.py::test_replicated_engine_with_arguments 3.97s call test_quota/test.py::test_add_remove_interval 3.95s teardown test_reload_query_masking_rules/test.py::test_reload_query_masking_rules 3.33s call test_partition/test.py::test_make_clone_in_detached 2.89s teardown test_reloading_storage_configuration/test.py::test_update_disk 2.74s call test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_all_database_tables 2.50s call test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids 2.48s call test_postgresql_replica_database_engine_1/test.py::test_replica_identity_index 2.42s teardown test_replicated_database_cluster_groups/test.py::test_cluster_groups 2.33s teardown test_replicated_database_alter_modify_order_by/test.py::test_alter_modify_order_by 2.23s call test_postgresql_replica_database_engine_1/test.py::test_changing_replica_identity_value 2.14s call test_replicated_merge_tree_encrypted_disk/test.py::test_different_keys 2.07s call test_replicated_merge_tree_encrypted_disk/test.py::test_same_keys 1.96s call test_quota/test.py::test_reload_users_xml_by_timer 1.95s call test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces 1.95s teardown test_postgresql_replica_database_engine_1/test.py::test_concurrent_transactions 1.75s call test_replicated_merge_tree_encryption_codec/test.py::test_same_keys 1.73s call test_partition/test.py::test_detached_part_dir_exists 1.70s call test_quota/test.py::test_add_remove_quota 1.54s call test_quota/test.py::test_dcl_introspection 1.50s call test_quota/test.py::test_dcl_management 1.46s teardown test_postgresql_replica_database_engine_1/test.py::test_virtual_columns 1.21s call test_partition/test.py::test_drop_detached_parts 1.14s setup test_partition/test.py::test_drop_detached_parts 1.08s call test_replicated_database_alter_modify_order_by/test.py::test_alter_modify_order_by 1.02s call test_partition/test.py::test_partition_complex 1.02s call test_part_log_table/test.py::test_config_disk_name_test 1.00s teardown test_postgresql_replica_database_engine_1/test.py::test_abrupt_connection_loss_while_heavy_replication 0.98s teardown test_postgresql_replica_database_engine_1/test.py::test_changing_replica_identity_value 0.98s call test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[0-16] 0.98s call test_quota/test.py::test_exceed_quota 0.92s teardown test_postgresql_replica_database_engine_1/test.py::test_abrupt_server_restart_while_heavy_replication 0.89s call test_partition/test.py::test_attach_check_all_parts 0.88s call test_replicated_merge_tree_encryption_codec/test.py::test_different_keys 0.84s call test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[8192-12] 0.80s call test_quota/test.py::test_tracking_quota 0.78s call test_replicated_merge_tree_s3_zero_copy/test.py::test_s3_check_restore 0.77s call test_postgresql_replica_database_engine_1/test.py::test_virtual_columns 0.72s teardown test_postgresql_replica_database_engine_1/test.py::test_multiple_databases 0.71s teardown test_postgresql_replica_database_engine_1/test.py::test_clickhouse_restart 0.62s call test_reload_query_masking_rules/test.py::test_reload_query_masking_rules 0.61s call test_quota/test.py::test_query_inserts 0.60s setup test_quota/test.py::test_users_xml_is_readonly 0.58s call test_quota/test.py::test_quota_from_users_xml 0.58s call test_postgresql_replica_database_engine_1/test.py::test_quoting_2 0.56s call test_part_log_table/test.py::test_config_with_non_standard_part_log 0.52s call test_quota/test.py::test_simpliest_quota 0.51s call test_postgresql_replica_database_engine_1/test.py::test_quoting_1 0.49s call test_part_log_table/test.py::test_config_without_part_log 0.47s setup test_quota/test.py::test_consumption_of_show_clusters 0.46s call test_replicated_engine_arguments/test.py::test_replicated_engine_with_arguments 0.46s call test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings 0.46s call test_part_log_table/test.py::test_config_with_standard_part_log 0.44s teardown test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[8192-12] 0.42s setup test_quota/test.py::test_consumption_of_show_databases 0.41s setup test_quota/test.py::test_reload_users_xml_by_timer 0.39s teardown test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[0-16] 0.38s setup test_quota/test.py::test_tracking_quota 0.38s teardown test_replicated_merge_tree_s3_zero_copy/test.py::test_drop_table 0.37s teardown test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_all_database_tables 0.37s teardown test_postgresql_replica_database_engine_1/test.py::test_table_schema_changes 0.33s setup test_partition/test.py::test_partition_complex 0.33s teardown test_postgresql_replica_database_engine_1/test.py::test_replicating_dml 0.33s setup test_quota/test.py::test_consumption_of_show_processlist 0.33s teardown test_postgresql_replica_database_engine_1/test.py::test_single_transaction 0.32s setup test_quota/test.py::test_consumption_of_show_privileges 0.32s teardown test_postgresql_replica_database_engine_1/test.py::test_restart_server_while_replication_startup_not_finished 0.32s setup test_quota/test.py::test_query_inserts 0.32s teardown test_postgresql_replica_database_engine_1/test.py::test_replica_identity_index 0.32s teardown test_postgresql_replica_database_engine_1/test.py::test_many_concurrent_queries 0.31s setup test_quota/test.py::test_add_remove_quota 0.31s call test_part_log_table/test.py::test_part_log_contains_partition 0.31s teardown test_postgresql_replica_database_engine_1/test.py::test_quoting_2 0.31s setup test_quota/test.py::test_simpliest_quota 0.29s setup test_partition/test.py::test_system_detached_parts 0.29s setup test_quota/test.py::test_exceed_quota 0.29s teardown test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_subset_of_database_tables 0.28s setup test_quota/test.py::test_dcl_introspection 0.28s setup test_quota/test.py::test_quota_from_users_xml 0.28s teardown test_postgresql_replica_database_engine_1/test.py::test_quoting_1 0.27s teardown test_postgresql_replica_database_engine_1/test.py::test_different_data_types 0.27s setup test_quota/test.py::test_consumption_of_show_tables 0.26s teardown test_postgresql_replica_database_engine_1/test.py::test_drop_database_while_replication_startup_not_finished 0.25s setup test_quota/test.py::test_dcl_management 0.25s teardown test_postgresql_replica_database_engine_1/test.py::test_user_managed_slots 0.23s call test_quota/test.py::test_consumption_of_show_processlist 0.23s setup test_partition/test.py::test_partition_simple 0.20s call test_partition/test.py::test_partition_simple 0.20s setup test_partition/test.py::test_cannot_attach_active_part 0.19s call test_partition/test.py::test_cannot_attach_active_part 0.18s call test_quota/test.py::test_consumption_of_show_privileges 0.17s call test_remote_prewhere/test.py::test_remote 0.14s call test_quota/test.py::test_consumption_of_show_clusters 0.13s call test_quota/test.py::test_consumption_of_show_databases 0.13s teardown test_partition/test.py::test_attach_check_all_parts 0.13s call test_quota/test.py::test_consumption_of_show_tables 0.13s teardown test_partition/test.py::test_drop_detached_parts 0.07s teardown test_partition/test.py::test_partition_simple 0.07s teardown test_partition/test.py::test_cannot_attach_active_part 0.07s call test_quota/test.py::test_users_xml_is_readonly 0.04s teardown test_partition/test.py::test_partition_complex 0.00s teardown test_reload_client_certificate/test.py::test_correct_cn_cert 0.00s teardown test_quota/test.py::test_add_remove_interval 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_all_database_tables 0.00s teardown test_replicated_mutations/test.py::test_mutations 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_single_transaction 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_clickhouse_restart 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_virtual_columns 0.00s teardown test_reloading_storage_configuration/test.py::test_remove_disk_from_policy 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_user_managed_slots 0.00s teardown test_reloading_storage_configuration/test.py::test_remove_policy 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_quoting_1 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_many_concurrent_queries 0.00s setup test_partition/test.py::test_detached_part_dir_exists 0.00s teardown test_replicated_merge_tree_encrypted_disk/test.py::test_different_keys 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_restart_server_while_replication_startup_not_finished 0.00s teardown test_reloading_storage_configuration/test.py::test_remove_volume_from_policy 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_abrupt_server_restart_while_heavy_replication 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_replicating_dml 0.00s teardown test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes0] 0.00s teardown test_reloading_storage_configuration/test.py::test_remove_disk 0.00s setup test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[8192-12] 0.00s setup test_rename_column/test.py::test_rename_with_parallel_insert 0.00s setup test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[0-16] 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_quoting_2 0.00s teardown test_rename_column/test.py::test_rename_with_parallel_merges 0.00s teardown test_partition/test.py::test_make_clone_in_detached 0.00s setup test_rename_column/test.py::test_rename_with_parallel_ttl_move 0.00s setup test_rename_column/test.py::test_rename_parallel 0.00s teardown test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table 0.00s teardown test_rename_column/test.py::test_rename_distributed 0.00s setup test_reloading_storage_configuration/test.py::test_remove_volume_from_policy 0.00s setup test_replicated_merge_tree_encryption_codec/test.py::test_same_keys 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_changing_replica_identity_value 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_different_data_types 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_table_schema_changes 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_drop_database_while_replication_startup_not_finished 0.00s setup test_rename_column/test.py::test_rename_with_parallel_select 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_replica_identity_index 0.00s teardown test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server 0.00s setup test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server 0.00s teardown test_rename_column/test.py::test_rename_parallel_same_node 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_subset_of_database_tables 0.00s teardown test_quota/test.py::test_exceed_quota 0.00s setup test_replicated_merge_tree_s3_zero_copy/test.py::test_s3_check_restore 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_concurrent_transactions 0.00s setup test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table 0.00s teardown test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids 0.00s setup test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes0] 0.00s setup test_part_log_table/test.py::test_config_without_part_log 0.00s setup test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes1] 0.00s teardown test_reloading_storage_configuration/test.py::test_add_disk 0.00s teardown test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table 0.00s teardown test_rename_column/test.py::test_rename_with_parallel_ttl_delete 0.00s setup test_reloading_storage_configuration/test.py::test_add_disk_to_policy 0.00s setup test_reloading_storage_configuration/test.py::test_add_volume_to_policy 0.00s setup test_reloading_storage_configuration/test.py::test_remove_disk_from_policy 0.00s setup test_postgresql_replica_database_engine_1/test.py::test_multiple_databases 0.00s teardown test_reloading_storage_configuration/test.py::test_add_disk_to_policy 0.00s teardown test_quota/test.py::test_tracking_quota 0.00s setup test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch 0.00s setup test_rename_column/test.py::test_rename_parallel_same_node 0.00s setup test_replicated_merge_tree_encrypted_disk/test.py::test_same_keys 0.00s setup test_rename_column/test.py::test_rename_with_parallel_slow_insert 0.00s teardown test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select 0.00s setup test_rename_column/test.py::test_rename_with_parallel_ttl_delete 0.00s setup test_part_log_table/test.py::test_part_log_contains_partition 0.00s setup test_reloading_storage_configuration/test.py::test_remove_disk 0.00s teardown test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server 0.00s setup test_part_log_table/test.py::test_config_with_non_standard_part_log 0.00s teardown test_rename_column/test.py::test_rename_with_parallel_select 0.00s setup test_rename_column/test.py::test_rename_with_parallel_merges 0.00s teardown test_part_log_table/test.py::test_config_disk_name_test 0.00s setup test_part_log_table/test.py::test_config_with_standard_part_log 0.00s teardown test_quota/test.py::test_dcl_introspection 0.00s teardown test_quota/test.py::test_consumption_of_show_tables 0.00s setup test_replicated_merge_tree_encryption_codec/test.py::test_different_keys 0.00s setup test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server 0.00s setup test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select 0.00s setup test_partition/test.py::test_make_clone_in_detached 0.00s teardown test_quota/test.py::test_dcl_management 0.00s teardown test_quota/test.py::test_simpliest_quota 0.00s teardown test_part_log_table/test.py::test_config_with_standard_part_log 0.00s teardown test_rename_column/test.py::test_rename_with_parallel_insert 0.00s setup test_reloading_storage_configuration/test.py::test_update_disk 0.00s setup test_reloading_storage_configuration/test.py::test_remove_policy 0.00s teardown test_rename_column/test.py::test_rename_parallel 0.00s setup test_reloading_storage_configuration/test.py::test_add_disk_to_separate_config 0.00s teardown test_quota/test.py::test_quota_from_users_xml 0.00s setup test_reload_client_certificate/test.py::test_wrong_cn_cert 0.00s setup test_reloading_storage_configuration/test.py::test_new_policy_works 0.00s setup test_reloading_storage_configuration/test.py::test_add_policy 0.00s setup test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces 0.00s teardown test_quota/test.py::test_query_inserts 0.00s teardown test_rename_column/test.py::test_rename_with_parallel_slow_insert 0.00s teardown test_reloading_storage_configuration/test.py::test_new_policy_works 0.00s teardown test_reloading_storage_configuration/test.py::test_add_disk_to_separate_config 0.00s teardown test_quota/test.py::test_add_remove_quota 0.00s teardown test_quota/test.py::test_consumption_of_show_clusters 0.00s teardown test_reloading_storage_configuration/test.py::test_add_policy 0.00s teardown test_quota/test.py::test_reload_users_xml_by_timer 0.00s teardown test_quota/test.py::test_consumption_of_show_databases 0.00s teardown test_quota/test.py::test_consumption_of_show_privileges 0.00s teardown test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces 0.00s teardown test_part_log_table/test.py::test_config_with_non_standard_part_log 0.00s teardown test_quota/test.py::test_consumption_of_show_processlist 0.00s teardown test_reloading_storage_configuration/test.py::test_add_volume_to_policy 0.00s teardown test_part_log_table/test.py::test_config_without_part_log 0.00s teardown test_partition/test.py::test_detached_part_dir_exists 0.00s teardown test_replicated_merge_tree_encryption_codec/test.py::test_different_keys =========================== short test summary info ============================ PASSED test_part_log_table/test.py::test_config_disk_name_test PASSED test_part_log_table/test.py::test_config_with_non_standard_part_log PASSED test_part_log_table/test.py::test_config_with_standard_part_log PASSED test_partition/test.py::test_attach_check_all_parts PASSED test_part_log_table/test.py::test_config_without_part_log PASSED test_part_log_table/test.py::test_part_log_contains_partition PASSED test_partition/test.py::test_cannot_attach_active_part PASSED test_partition/test.py::test_detached_part_dir_exists PASSED test_partition/test.py::test_drop_detached_parts PASSED test_quota/test.py::test_add_remove_interval PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_keys PASSED test_quota/test.py::test_add_remove_quota PASSED test_quota/test.py::test_consumption_of_show_clusters PASSED test_quota/test.py::test_consumption_of_show_databases PASSED test_partition/test.py::test_make_clone_in_detached PASSED test_reloading_storage_configuration/test.py::test_add_disk PASSED test_quota/test.py::test_consumption_of_show_privileges PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces PASSED test_quota/test.py::test_consumption_of_show_processlist PASSED test_quota/test.py::test_consumption_of_show_tables PASSED test_partition/test.py::test_partition_complex PASSED test_rename_column/test.py::test_rename_distributed PASSED test_partition/test.py::test_partition_simple PASSED test_replicated_merge_tree_encryption_codec/test.py::test_same_keys PASSED test_quota/test.py::test_dcl_introspection PASSED test_quota/test.py::test_dcl_management PASSED test_reloading_storage_configuration/test.py::test_add_disk_to_policy PASSED test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table PASSED test_quota/test.py::test_exceed_quota PASSED test_quota/test.py::test_query_inserts PASSED test_partition/test.py::test_system_detached_parts PASSED test_quota/test.py::test_quota_from_users_xml PASSED test_reloading_storage_configuration/test.py::test_add_disk_to_separate_config PASSED test_quota/test.py::test_reload_users_xml_by_timer PASSED test_quota/test.py::test_simpliest_quota PASSED test_quota/test.py::test_tracking_quota PASSED test_quota/test.py::test_users_xml_is_readonly PASSED test_reloading_storage_configuration/test.py::test_add_policy PASSED test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server PASSED test_rename_column/test.py::test_rename_distributed_parallel_insert_and_select PASSED test_reloading_storage_configuration/test.py::test_add_volume_to_policy PASSED test_postgresql_replica_database_engine_1/test.py::test_abrupt_connection_loss_while_heavy_replication PASSED test_reloading_storage_configuration/test.py::test_new_policy_works PASSED test_rename_column/test.py::test_rename_parallel PASSED test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table PASSED test_replicated_mutations/test.py::test_mutations PASSED test_reloading_storage_configuration/test.py::test_remove_disk PASSED test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop PASSED test_rename_column/test.py::test_rename_parallel_same_node PASSED test_reloading_storage_configuration/test.py::test_remove_disk_from_policy PASSED test_replicated_merge_tree_s3_zero_copy/test.py::test_drop_table PASSED test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[0-16] PASSED test_replicated_merge_tree_s3_zero_copy/test.py::test_insert_select_replicated[8192-12] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server PASSED test_reloading_storage_configuration/test.py::test_remove_policy PASSED test_rename_column/test.py::test_rename_with_parallel_insert PASSED test_reload_client_certificate/test.py::test_correct_cn_cert PASSED test_replicated_merge_tree_s3_zero_copy/test.py::test_s3_check_restore PASSED test_reloading_storage_configuration/test.py::test_remove_volume_from_policy PASSED test_reload_query_masking_rules/test.py::test_reload_query_masking_rules PASSED test_postgresql_replica_database_engine_1/test.py::test_abrupt_server_restart_while_heavy_replication PASSED test_reloading_storage_configuration/test.py::test_update_disk PASSED test_postgresql_replica_database_engine_1/test.py::test_changing_replica_identity_value PASSED test_rename_column/test.py::test_rename_with_parallel_merges PASSED test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch PASSED test_rename_column/test.py::test_rename_with_parallel_select PASSED test_reload_zookeeper/test.py::test_reload_zookeeper PASSED test_postgresql_replica_database_engine_1/test.py::test_clickhouse_restart PASSED test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes0] PASSED test_replicated_engine_arguments/test.py::test_replicated_engine_with_arguments PASSED test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper PASSED test_rename_column/test.py::test_rename_with_parallel_slow_insert PASSED test_remote_prewhere/test.py::test_remote PASSED test_postgresql_replica_database_engine_1/test.py::test_concurrent_transactions PASSED test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings PASSED test_postgresql_replica_database_engine_1/test.py::test_different_data_types PASSED test_rename_column/test.py::test_rename_with_parallel_ttl_delete PASSED test_postgresql_replica_database_engine_1/test.py::test_drop_database_while_replication_startup_not_finished PASSED test_replicated_database_alter_modify_order_by/test.py::test_alter_modify_order_by PASSED test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_all_database_tables PASSED test_postgresql_replica_database_engine_1/test.py::test_load_and_sync_subset_of_database_tables PASSED test_rename_column/test.py::test_rename_with_parallel_ttl_move PASSED test_replicated_mutations/test.py::test_mutations_dont_prevent_merges[nodes1] PASSED test_postgresql_replica_database_engine_1/test.py::test_many_concurrent_queries PASSED test_postgresql_replica_database_engine_1/test.py::test_multiple_databases PASSED test_postgresql_replica_database_engine_1/test.py::test_quoting_1 PASSED test_postgresql_replica_database_engine_1/test.py::test_quoting_2 PASSED test_postgresql_replica_database_engine_1/test.py::test_replica_identity_index PASSED test_postgresql_replica_database_engine_1/test.py::test_replicating_dml PASSED test_replicated_database_cluster_groups/test.py::test_cluster_groups PASSED test_postgresql_replica_database_engine_1/test.py::test_restart_server_while_replication_startup_not_finished PASSED test_postgresql_replica_database_engine_1/test.py::test_single_transaction PASSED test_postgresql_replica_database_engine_1/test.py::test_table_schema_changes PASSED test_replicated_merge_tree_compatibility/test.py::test_replicated_merge_tree_defaults_compatibility PASSED test_postgresql_replica_database_engine_1/test.py::test_user_managed_slots PASSED test_postgresql_replica_database_engine_1/test.py::test_virtual_columns PASSED test_reload_client_certificate/test.py::test_wrong_cn_cert PASSED test_replicated_merge_tree_encrypted_disk/test.py::test_different_keys PASSED test_replicated_merge_tree_encrypted_disk/test.py::test_same_keys ================= 100 passed, 43 warnings in 309.24s (0:05:09) =================